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PREDICTING  MILITARY  RECRUITER  EFFECTIVENESS:  A  LITERATURE  REVIEW 


EXECUTIVE  SUMMARY 


Requirement: 

The  U.S.  Army  has  a  continuing  need  to  recruit  sufficient  numbers  of 
qualified  men  and  women  into  service.  Recruiters  responsible  for 
contacting  prospective  recruits  and  selling  them  on  an  Army  career  are 
clearly  key  personnel  fo^  enlisting  qualified  persons.  The  selection  of 
recruiters  with  strong  potential  for  success  on  this  job  is,  therefore, 
important.  The  purposes  of  the  present  report  are  three-fold:  (1)  to 
provide  information  on  the  individual  differences  variables  likely  to  be 
successful  predictors  of  military  recruiter  performance;  (2)  to  discuss 
issues  related  to  measuring  recruiter  performance;  and  (3)  to  identify 
research  on  civilian  sales  selection  that  is  relevant  to  recruiter 
selection  research. 


Procedure: 

Past  research  on  military  recruiter  selection  and  on  recruiter  perfor¬ 
mance  measurement  is  reviewed.  The  results  are  compared  to  similar 
research  for  sales  jobs  in  the  civilian  sector. 


Findings: 

Three  major  findings  arose  from  analysis  of  recruiter  and  civilian 
sales  selection  research. 

First,  non-cognitive  measures  have  successfully  predicted  military 
recruiter  performance.  More  specifically,  past  research  suggests  that 
vocational  interest  and  personality  variables  (such  as  dominance,  self- 
confidence,  and  spontaneity)  are  significantly  associated  with  military 
recruiter  performance.  Cognitive  variables  such  as  verbal  ability  and 
general  aptitude  have  been  employed  less  frequently  in  recruiter  selection 
research  and  appear  to  have  little  validity  for  predicting  military  re¬ 
cruiter  success. 

Second,  military  recruiter  performance  has  been  evaluated  using  super¬ 
visor,  peer,  and  self  ratings,  production  data,  and  measures  of  training 
success.  Each  type  of  criterion  has  advantages  and  disadvantages. 


Although  recruiter  productivity  (i.e.,  the  gross  number  of  recruits  over  a 
specified  time  frame)  is  an  obvious  performance  criterion,  production  data 
may  be  influenced  or  contaminated  by  environmental  factors  outside  the 
recruiter’s  control  such  as  the  unemployment  rate  in  the  territory. 
Additionally,  production  is  an  index  that  may  not  tap  some  aspects  of 
performance.  In  contrast,  peer,  self,  and  supervisory  ratings  are  probably 
not  restricted  by  environmental  factors  and  can  provide  multidimensional 
evaluations  of  performance.  These  features  offer  a  distinct  advantage  in 
selection  research.  On  the  other  hand,  the  quality  of  such  subjective 
ratings  is  influenced  by  the  source  of  the  rating  and  various  rating  errors 
and  biases.  Training  criteria  are  different  conceptually,  and  often 
methodologically,  from  measures  of  success  on-the-job.  That  is,  training 
performance  may  be  related  to  behaviors  such  as  studying  that  may  not  be 
highly  related  to  on-the-job  performance.  In  turn,  variables  that  predict 
job  performance  may  differ  from  those  that  are  related  to  training  success. 

Finally,  although  military  recruiting  is  essentially  a  sales  job,  the 
type  of  "product"  sold  distinguishes  this  job  from  sales  jobs  in  the 
civilian  sector.  In  particular,  recruiters  are  selling  a  career  or  life¬ 
style  rather  than  material  goods  or  services.  Research  in  the  civilian 
sector  suggests  that  differences  in  the  type  of  product  sold  moderates 
predictor-criterion  relationships.  Even  though  "product"  type  differences 
should  produce  different  results  for  military  recruiter  and  civilian  sales 
jobs,  there  are  two  major  consistencies  between  factors  that  contribute  to 
military  recruiter  and  civilian  sales  success.  First,  personality  vari¬ 
ables  such  as  dominance  have  yielded  reasonable  validities  in  both  arenas. 
Second,  aptitude  and  verbal  ability  measures  have  shown  little  merit  for 
predicting  either  civilian  sales  or  military  recruiter  performance. 
Additionally,  research  from  the  civilian  sector  and,  to  a  limited  extent, 
military  recruiter  selection  research  suggests  that  skill  level  variables 
such  as  assessment  center  scores  may  be  useful  predictors  of  recruiter  job 
performance. 


Utilization  of  Findings: 

The  present  report  provides  a  review  of  all  previous  military  recruiter 
selection  research.  Moreover,  the  information  provided  in  this  report  can 
be  used  for  at  least  three  purposes:  (1)  to  help  identify  variables  likely 
to  be  successful  predictors  in  future  selection  research;  (2)  to  provide 
guidance  in  criterion  development  work  related  to  the  Army  recruiter  job; 
and  (3)  to  aid  in  identifying  factors  likely  to  influence  or  contaminate 
production  data.  In  sum,  researchers  contemplating  future  work  in  this 
area  should  first  carefully  review  the  issues  in  military  recruiter 
criterion  measurement  and  the  conclusions  about  predictors  of  military 
recruiter  performance  presented  in  this  report. 


PREDICTING  MILITARY  RECRUITER  EFFECTIVENESS: 
A  LITERATURE  REVIEW 


Introduction 


Over  the  past  20  years,  the  U.S.  Armed  Forces’  all -volunteer  concept 
has  placed  considerable  pressure  on  recruiters  to  attract  sufficient  num¬ 
bers  of  qualified  recruits.  Recruiters  can  no  longer  depend  on  selective 
service  to  help  them  fill  personnel  quotas,  and  must  instead  sell  military 
service  as  an  alternative  to  civilian  job  opportunities.  Additionally, 
military  recruiters  must  attract  to  the  service  individuals  who  either 
already  possess  technical  skills  or  have  good  potential  for  developing 
skills  necessary  to  do  technical  jobs.  The  percentage  of  military  per¬ 
sonnel  who  must  be  technically  proficient  is  growing  and  will  continue  to 
grow  in  the  future  as  the  services  continue  to  modernize  their  equipment 
and  jobs.  Thus,  effective  recruitment  requires  enlisting  sufficient 
numbers  of  high  quality  recruits. 

An  important  question  for  the  military  services,  therefore,  is  how  to 
maintain  an  effective  recruiting  force.  Sales  and  other  kinds  of  training 
is  clearly  one  way  to  enhance  and  upgrade  the  performance  of  recruiters. 
Another  vehicle  is  selection  of  individuals  who  possess  the  requisite 
skills,  abilities,  and  personal  characteristics  to  become  successful  mili¬ 
tary  recruiters.  Regarding  selection,  the  Army,  Navy,  Marine  Corps,  and 
Air  Force  have  conducted  a  number  of  studies  to  identify  tests  and  other 
kinds  of  measures  that  might  be  used  to  predict  success  in  military  re¬ 
cruiting.  The  present  paper  is  a  review  of  this  work. 

Research  on  military  recruiter  selection  has  typically  involved  admin¬ 
istering  to  a  sample  of  recruiters  one  or  more  predictor  measures,  such  as 
ability  tests,  personality  scales,  or  vocational  interest  inventories  and 
also  assessing  in  some  manner  the  performance  of  these  same  recruiters. 
Relationships  between  the  predictor  and  performance  criterion  measures  have 
then  been  used  to  evaluate  the  validity  of  the  predictors.  In  most  cases, 
a  concurrent  validation  strategy  has  been  employed  in  which  predictor 
measures  are  administered  to  recruiters  who  are  already  on  the  job  and  at 
or  about  the  same  time  the  criterion  measure(s)  is  taken. 

As  mentioned,  selection  research  on  military  recruiters  has  involved 
several  different  kinds  of  predictor  measures.  Also,  different  types  of 
performance  criteria  have  been  employed  in  this  work.  Performance  ratings 
or  nominations  made  by  supervisors  or  peers,  production  data  (i.e.,  how 
many  recruits  per  unit  time  are  brought  into  the  service),  and  training 
performance  criteria  have  all  been  used  to  assess  the  validity  of  predictor 
measures.  The  present  report  is  organized  according  to  these  three  dif¬ 
ferent  kinds  of  criteria.  Each  chapter  reviews  and  evaluates  the  validity 
of  the  various  kinds  of  predictors  against  one  of  these  criterion  types. 

The  reason  for  this  structure  is  that  validity  of  a  predictor  depends 
to  some  extent  on  the  particular  criterion  employed.  The  three  types  of 
criteria  are  sufficiently  different  in  nature  that  a  predictor  could  relate 
well  to  performance  utilizing  one  of  the  criteria  but  not  as  well  with 
another  criterion.  Further,  each  of  these  three  kinds  of  measures  has 


merit  as  a  criterion  and  is  therefore  worth  trying  to  predict.  Ratings  by 
superiors,  for  example,  reflect  informed  opinions  about  recruiter  perfor¬ 
mance  based  on  good  opportunity  to  observe  recruiter  performance-related 
behavior  over  a  period  of  time.  Production,  number  of  recruits  brought 
into  the  service  per  unit  time,  is  a  "bottom-line”  index  of  a  recruiter’s 
effectiveness,  and  success  in  training  is  clearly  important  as  a  prere¬ 
quisite  for  performing  effectively  later  on  the  job. 

To  obtain  information  on  military  recruiter  selection  research,  we 
contacted  representatives  of  the  four  branches  of  the  Armed  Services  and 
requested  copies  of  technical  reports.  Studies  are  discussed  chronologi¬ 
cally  within  each  chapter,  and  for  each  study  a  full  summary  is  provided 
including  a  description  of  the  predictor  measures  and  criteria  utilized  and 
an  evaluation  of  the  results  and  conclusions. 

The  purpose  of  this  review  is  to  explicate  the  merits  of  different 
types  of  predictor  measures  (e.g.,  cognitive,  non-cognitive)  for  predicting 
various  aspects  of  performance  in  recruiting.  Studies  reviewed  in  this 
paper  are  listed  according  to  the  types  of  predictors  and  criteria  employed 
in  Table  1,  and  these  predictor-criterion  relationships  are  summarized  at 
the  end  of  each  chapter.  The  final  chapter  of  this  paper  incorporates 
civilian  sales  selection  research  to  provide  a  broader  perspective  of 
factors  that  influence  effectiveness  in  the  military  recruiter  job. 


Commanding  Officer  Nominations  and  Supervisor  Ratings 
as  Measures  of  Recruiter  Performance 


In  several  studies,  researchers  have  obtained  nominations  of  effective 
and  ineffective  recruiters  from  commanding  officers  and  used  these  nomina¬ 
tions  as  criteria  for  assessing  the  validity  of  predictors.  Likewise, 
supervisors’  ratings  of  recruiter  effectiveness  have  been  gathered  and 
applied  as  criteria  for  predictor  validation.  This  section  describes 
studies  employing  commanding  officer  nominations  and/or  supervisor  ratings 
as  criteria  for  success  in  recruiting. 

Wollack  and  Kipnis  (19601.  Commanding  officer  nominations  and  super¬ 
visor  ratings  served  as  criteria  in  a  validation  and  cross-validation  study 
conducted  by  Wollack  and  Kipnis  (1960).  These  researchers  developed  a  Navy 
recruiter  selection  battery  composed  of  13  measures  designed  to  reflect 
interests  in  recruiting  activities,  fluency  of  expression,  general  apti¬ 
tude,  and  knowledge  of  the  Navy.  The  predictor  battery  included: 

1.  Fluency  of  Expression:  Four  timed  inventories  designed  to  measure  a 

subject’s  verbal  abilities.  (These  timed  inventories  were  administered 

to  the  cross-validation  group  only.) 

a.  Thing  Listing  Test  -  requires  the  subject  to  write  names  of  as 
many  things  as  possible. 

b.  First  Letters  Test  -  requires  the  subject  to  write  as  many  words 

as  he/she  can  think  of  beginning  with  the  letter  "b". 

c.  Four  Word  Combination  Test  -  requires  the  examinee  to  compose  four 

word  sentences  using  the  four  letters  presented  in  each  item  as 
the  initial  letters  of  the  words. 

d.  Inventive  Opposite  Test  -  a  word  is  presented  along  with  the  first 
letter  of  one  of  its  synonyms  and  the  first  letter  of  an  antonym. 
The  subject  is  required  to  fill  in  the  synonym  and  antonym  for 
each  word  in  the  series. 

2.  Knowledge  of  and  Enthusiasm  for  the  Navy 

a.  Navy  Knowledge  -  a  60-item  inventory  designed  to  measure  a  sub¬ 
ject’s  knowledge  of  Navy  history  and  traditions. 

b.  Career  Preference  Scale  -  30  items  intended  to  measure  the 
subject’s  attitude  toward  a  Navy  career. 

c.  Career  Motivation  Survey  -  a  39-item  attitude  scale,  measuring 
attitudes  toward  various  aspects  of  Navy  life. 

3.  Vocational  and  Sports  Interests 

a.  Kuder  Preference  Record  -  a  published  inventory  designed  to  mea¬ 
sure  a  subject’s  interest  in  ten  broad  occupational  areas. 


b.  Sports  Inventory  -  a  50- item  inventory  dealing  with  rules  and 

plays  of  various  sports,  designed  to  measure  a  subject’s  interest 
in  masculine  activities. 

4.  Basic  Test  Battery:  Measures  aptitude  in  four  areas: 

a.  General  Intelligence 

b.  Arithmetic  Reasoning 

c.  Mechanical  Comprehension 

d.  Clerical  Skills 

The  validation  sample  consisted  of  410  active  recruiters  representing 
40  recruiting  stations.  This  sample  was  formed  by  contacting  the 
commanding  officers  from  the  40  recruiting  stations  and  asking  each  to 
nominate  the  most  effective  and  least  effective  recruiters  from  their 
respective  stations.  These  nominations  were  employed  as  the  criterion 
measure  against  which  the  predictors  were  validated  (most  effective 
recruiter  group  N  =  205,  least  effective  recruiter  group  N  =  205). 

Item  analyses  were  conducted  for  The  Navy  Knowledge  Test,  Career  Pre¬ 
ference  Scale,  Career  Motivation  Survey,  Kuder  Preference  Record,  and 
Sports  Inventory  using  the  effective  and  ineffective  nominations  as  the 
criterion.  Items  that  discriminated  between  effective  and  ineffective 
recruiters  beyond  the  .20  level  of  confidence  were  retained  for  cross- 
validation.  Seven  items  from  the  Navy  Knowledge  Test  discriminated  beyond 
the  .20  level,  as  did  15  items  from  the  Career  Preference  Scale,  24  items 
from  the  Career  Motivation  Survey,  13  items  from  the  Sports  Inventory,  and 
four  scales  from  the  Kuder. 

Navy  service  rating  classification  (duty  prior  to  serving  as  a  re¬ 
cruiter)  was  also  analyzed,  and  significant  differences  were  found  to  exist 
between  the  effective  and  ineffective  groups.  The  effective  recruiter 
group  contained  larger  proportions  of  men  in  Deck  (jk.OI)  and  Aviation 
specialties  (jk.01).  The  ineffective  group  contained  significantly  greater 
proportions  of  men  in  Engineering  and  Hull  specialties  (jx.01)  and  in  Con¬ 
struction  jobs  (jx.02). 

The  sample  used  to  cross-val idate  the  predictor  battery  consisted  of 
260  students  attending  a  six-week  recruiter  course.  The  predictor  battery 
was  administered  to  the  cross-validation  group  while  they  were  attending 
the  course.  Approximately  one  year  after  the  initial  testing,  when  all 
students  had  been  assigned  to  recruiting  duty,  ratings  were  collected  from 
the  two  supervisors  most  familiar  with  each  recruiter.  The  evaluation 


*The  Navy  term  "station"  is  equivalent  to  the  Army  recruiting  "station" 
level.  Navy  districts  are  approximately  equivalent  to  the  battalion  level, 
and  the  Navy  term  "zone"  is  equivalent  to  the  Army  term  "company." 


forms  contained  four  14-point  scales:  Technical  Competence,  Willingness  to 
Work,  Military  Manner,  and  Adaptability.  The  forms  also  required  the 
supervisors  to  answer  the  following  two  questions  using  5-point  scales: 


1.  If  you  had  been  given  the  chance  would  you  have  recommended  the  man  for 

recruiting  duty? 

2.  Is  this  man  effective  in  recruiting  personnel? 

The  six  ratings  on  the  evaluation  scales  were  treated  as  six  different 
criterion  measures.  Recruiter  scores  on  each  dimension  were  computed  by 
adding  the  two  supervisory  ratings  together.  When  only  a  single  supervi¬ 
sory  rating  was  available,  the  dimension  scores  were  simply  doubled.  A 
total  of  222  recruiters  were  rated  by  two  supervisors  and  interrater  relia¬ 
bilities  ranged  from  .56-. 66,  indicating  an  acceptable  level  of  rater 
agreement. 

Product-moment  correlations  between  the  predictors  selected  previously 
and  the  criterion  measures  were  computed  using  the  cross-validation  sample 
of  260.  Three  of  the  variables  significantly  predicted  the  overall  evalua¬ 
tion  of  recruiting  performance:  The  Kuder  Persuasive  Scale  (r  -  .24, 
fi<.01),  the  Kuder  Scientific  Scale  (r  -  -.17,  fi<.01),  and  the  Career  Moti¬ 
vation  Survey  item  analysis  key  (r  -  .13,  b<.05).  None  of  the  other 
predictors  correlated  significantly  with  any  of  the  criteria  with  the 
exception  of  one  fluency  of  expression  measure,  the  First  Letters  Test, 
which  yielded  a  correlation  of  -.18  (gc.Ol)  with  Military  Manner.  A 
multiple  correlation  was  computed  using  the  Persuasive,  Scientific,  and 
Career  Motivation  survey  predictors  against  the  criterion  measure  of 
overall  recruiter  effectiveness.  The  result  (r  -  .26)  was  not  signifi¬ 
cantly  greater  than  the  correlation  between  that  performance  dimension  and 
the  Persuasive  Scale  alone.  Thus,  results  of  the  cross-validation  analysis 
suggest  that  an  effective  Navy  recruiter  has  persuasive  interests,  is  not 
very  interested  in  scientific  pursuits,  and  believes  in  the  value  of  a  Navy 
career. 

However,  overall,  results  here  were  disappointing.  Very  few  of  the 
wide  variety  of  cognitive  and  non-cognitive  variables  correlated  signifi¬ 
cantly  with  performance.  Results  regarding  the  cognitive  predictors  were 
particularly  disappointing  in  that  only  one  out  of  24  predictor-criterion 
correlations  for  the  verbal  fluency  tests  was  significant,  and  none  of  the 
general  aptitude  tests  showed  evidence  of  validity. 

Krug  (19721.  In  another  study  (Krug,  1972),  both  biographical  and 
personality  items  were  included  in  a  battery  designed  to  predict  Navy 
recruiter  performance.  Krug  administered  to  Navy  enlisted  and  officer 
recruiters  the  16PF  personality  inventory,  a  25-item  inventory  supplement 
(including  a  7-item  motivational  distortion  scale  and  18  items  designed  to 
measure  strength  of  motivation  toward  a  career  as  a  Naval  recruiter),  and 
seven  items  tapping  demographic  information.  Commanding  officers  from  each 
of  41  Naval  recruiting  stations  across  the  continental  United  States  were 
asked  to  nominate  five  enlisted  recruiters  and  one  officer  recruiter  from 
among  the  top  50  percent  performers  and  five  enlisted  recruiters  and  one 
officer  recruiter  from  among  the  bottom  50  percent  performers  of  recruiters 
presently  on  duty.  These  nominations  of  effective  and  ineffective 
recruiters  served  as  the  performance  criterion  against  which  the  16PF, 


supplemental,  and  demographic  variables  were  validated  (N  =  383  enlisted 
recruiters;  N  =  74  officer  recruiters). 

A  stepwise  multiple  regression  of  the  predictor  variables  on  the 
criterion  resulted  in  a  multiple  correlation  coefficient  of  .40  for  the 
enlisted  sample,  significant  beyond  the  .01  level.  When  the  same  equation 
was  applied  to  the  officer  recruiter  cross-validation  sample,  a  multiple 
correlation  coefficient  of  .25  was  obtained  (e<.05).  According  to  these 
results,  the  effective  Navy  recruiter  is  typically  married,  has  more  years 
of  formal  education,  tends  to  be  warm  and  outgoing,  dominant,  aggressive, 
self-assured,  and  politically  conservative. 

Although  the  results  from  the  cross-validation  sample  supported  the 
validity  of  this  instrument,  it  is  important  to  take  a  closer  look  at  the 
regression  equation.  The  most  influential  variable,  Marital  Status,  was 
assigned  a  high  positive  weight  in  the  prediction  equation.  Yet,  only  13 
of  the  383  enlisted  recruiters  in  the  sample  were  unmarried.  In  a  selec¬ 
tion  situation,  a  low  base  rate  of  unmarried  recruiters  would  make  that 
variable  less  useful  than  a  variable  with  a  more  balanced  base  rate. 

Abrahams.  Neuman,  and  Rim! and  (19731.  Another  study  focusing  on  the 
utility  of  non-cognitive  predictors  for  selecting  Navy  recruiters  was 
conducted  by  Abrahams,  Neuman,  and  Rimland  (1973).  These  authors  employed 
the  Strong  Vocational  Interest  Blank  (SVIB)  and  an  officer  nomination 
criterion  to  develop  an  empirically  keyed  Recruiter  Interest  Scale  (RIS). 
The  commanding  officers  in  each  of  42  recruiting  districts  were  requested 
to  identify  the  five  least  effective  and  five  most  effective  recruiters, 
and  SVIBs  were  mailed  and  administered  to  each  of  these  420  recruiters. 

Complete  information  was  obtained  from  356  recruiters  representing  36 
of  the  recruiting  districts.  One  half  of  the  sample  was  used  to  develop 
the  RIS  while  the  other  half  was  used  as  the  cross-validation  group.  Each 
of  the  399  SVIB  items  was  weighted  according  to  the  proportion  of  most  and 
least  effective  recruiters  endorsing  the  response.  If  a  response  was 
endorsed  more  often  by  the  most  effective  recruiters  it  was  weighted  +1. 
Conversely,  if  a  response  was  endorsed  more  often  by  the  least  effective 
recruiters  it  was  weighted  -1.  The  115  items  with  the  largest  endorsement 
differences  were  included  in  the  SVIB-Recruiter  Interest  Scale  (RIS- 1 ) . 

When  the  "holdout"  group  was  used  to  cross-val idate  the  SVIB-RIS-1,  the 
scores  for  each  recruiter  were  tabulated  and  arranged  from  high  to  low 
(high  scores  indicating  potentially  effective  recruiter  performance  and  low 
scores  indicating  potentially  ineffective  recruiter  performance).  The 
cross-validation  sample  was  then  divided  into  quartiles  according  to  RIS 
scores.  The  top  quartile  (highest  RIS  scores)  contained  three  times  as  many 
effective  recruiters  as  did  the  bottom  quartile;  conversely,  the  bottom 
quartile  contained  three  times  as  many  ineffective  recruiters  as  did  the 
top  RIS  group.  Based  on  these  results,  the  authors  recommended  using  the 
SVIB-RIS-1  as  a  selection  device  to  exclude  low  scoring  individuals  from 
recruiter  service. 

The  authors  recommended  using  supplemental  inventories  (e.g.,  bio¬ 
graphical  items  found  effective  in  predicting  sales  performance)  to  in¬ 
crease  validity.  Abrahams  et  al .  also  stressed  that  a  better  criterion  of 
recruiter  effectiveness  must  be  developed  and  that  specific  elements  of 
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recruiter  performance  (e.g.,  prospecting,  closing  the  sale)  should  be  taken 
into  account  as  criteria  in  future  validity  studies. 

Brown.  Wood,  and  Harris  (1975).  Brown,  Wood,  and  Harris  developed  an 
Army  recruiter  selection  battery  composed  of  inventories  expected  to  assess 
personal  characteristics  thought  to  be  important  for  success  as  a  recruit¬ 
er.  The  selection  battery  consisted  of  15  paper-and-pencil  inventories  and 
one  verbal  performance  test.  Below  is  a  list  of  the  measures  included  in 
the  battery. 

1.  Verbal  Fluency- -Recruiters  were  asked  to  simulate  a  presentation  of  the 
benefits  of  Army  life  to  a  prospective  enlistee.  Each  presentation  was 
recorded  and  scored  by  computing  the  ratio  of  the  number  of  words 
spoken  during  the  first  two  minutes  of  the  presentation  to  the  number 
of  "ahs"  spoken  during  the  same  length  of  time. 

2.  Sociability  Heasures--four  inventories  were  used  to  measure  a  recruit¬ 
er’s  sociability  and  affiliative  tendency. 

3.  Achievement  -<tivation--Three  inventories  were  employed  to  tap  the 
tendency  to  ».  ”k  hard  to  achieve  self-appointed  goals. 

4.  Empathy  Measures- -Four  instruments  were  used  to  measure  the  ability  to 
understand  the  point  of  view  of  others  and  the  drive  to  win  or  complete 
a  sale. 

5.  Rejection  Tolerance  Measure--One  paper-and-pencil  inventory  was  em¬ 
ployed  to  measure  a  recruiter’s  tolerance  to  rejection,  rebuffs,  and 
insults. 

6.  Responsibility  and  Maturity  Measures--Three  instruments  were  used  to 
tap  information  about  a  recruiter’s  ability  to  manage  his  personal  and 
financial  duties. 

To  supplement  the  predictor  battery,  aptitude  test  scores  and  bio¬ 
graphical  information  were  gathered  from  Army  personnel  files. 

As  a  criterion  performance  index,  Brown  et  al .  devised  a  Composite 
Supervisory  Rating  procedure  to  select  two  extreme  groups  of  recruiters-- 
the  highly  successful  and  the  highly  unsuccessful.2  The  highest  ranking 
District  Recruiting  Command  (DRC)  and  the  lowest  ranking  DRC  (based  on 
objectives  achieved)  were  selected  from  each  of  the  five  Regional  Re¬ 
cruiting  Commands.  Five  supervisors  from  each  DRC  were  then  asked  to 
nominate  the  ten  best  and  ten  poorest  recruiters  within  their  DRC.  The 


2These  authors  also  developed  an  adjusted  production  criterion  (as 
described  in  the  next  section);  however,  when  the  validation  study  began, 
the  development  of  the  production  criterion  was  not  completed.  Super¬ 
visors’  ratings  were  thus  employed  as  criteria. 


8 


five  recruiters  mentioned  most  often  as  the  best  recruiters  were  used  in 
the  High  Criterion  Group,  while  the  five  recruiters  mentioned  most  often  as 
the  poorest  recruiters  were  placed  in  the  Low  Criterion  Group  (N  =  45  for 
the  High  Criterion  Group,  and  N  *  43  for  the  Low  Criterion  Group). 

The  4-  to  6-hour  battery  was  administered  to  the  High  and  Low  Criterion 
Groups  in  each  DRC,  and  additional  information  concerning  each  recruiter’s 
race,  religion,  and  aptitude  scores  was  collected  from  Army  personnel 
files.  Means  for  the  keyed  inventories  and  the  aptitude  scores  were  com¬ 
puted  separately  for  the  High  and  Low  Criterion  Groups.  None  of  the  per¬ 
sonality  measures  or  aptitude  scores  differentiated  significantly  between 
the  two  groups.  Twenty  background  items  pertaining  to  work  habits,  styles 
of  handling  finances  and  debts,  educational  background,  and  reaction  to 
challenging  or  stressful  situations  did  appear  to  distinguish  the  two 
groups;  however,  the  significance  levels  were  in  general  greater  than  .05 
for  these  items. 

Brown  et  al .  suggested  that  these  recruiters  were  a  relatively  homo¬ 
geneous  group,  because  all  were  required  to  meet  several  minimum  qualifica¬ 
tions  (e.g.,  GCT  scores,  age,  and  rank)  and  had  an  average  length  of 
service  of  fourteen  years.  Because  of  this,  the  recruiters  may  have  formed 
similar  attitudes  and  opinions,  and  thereby  limited  the  variance  in  atti¬ 
tude,  personal  preference,  and  personality  inventory  scores.  The  low 
validities,  therefore,  may  have  been  due  to  restriction  in  range. 

Graf  and  Bower  (1976) .  Building  on  the  Abrahams  et  al .  work,  Graf  and 
Bower  (1976)  investigated  the  usefulness  of  the  Strong  Vocational  Interest 
Blank  and  the  Navy  RIS  scales  for  selecting  Marine  Corps  recruiters.  The 
SVIB  was  administered  to  each  of  98  recruiters  working  in  the  Los  Angeles 
and  Santa  Ana  areas;  of  these,  77  inventories  were  completed  and  returned. 
Criterion  measures  consisted  of  officer  performance  ratings  using  a  3-point 
scale  for  below-average,  average,  and  above-average  recruiter  performance. 

Graf  and  Bower  (1976)  recognized  that  the  Navy  Recruiter  Interest  Scale 
(RIS)  developed  on  a  sample  of  Navy  recruiters  might  also  predict  Marine 
Corps  recruiter  performance,  and  scored  the  Marine  Corps  recruiter  re¬ 
sponses  on  the  RIS  in  order  to  evaluate  the  scale’s  validity.  Results  were 
in  the  predicted  direction  (a  t-test  significant  beyond  the  .05  level)  for 
a  comparison  of  the  upper  and  lower  35  percent  of  the  RIS  score  distribu¬ 
tion.  In  addition,  the  higher  the  RIS  score,  the  greater  the  likelihood 
that  the  recruiter  was  in  the  effective  group.  Recruiters  were  then  dicho¬ 
tomized  as  above-average  or  below-average  according  to  the  supervisory 
field  ratings,  and  the  two  groups’  RIS  scores  were  compared.  A  significant 
difference  (beyond  the  .05  level)  in  the  predicted  direction  (r/pt.  bi¬ 
serial  «  .30,  jk.05)  was  found. 

Although  the  RIS  effectively  discriminated  between  above-average  and 
below-average  Marine  Corps  recruiters,  Graf  and  Bower  hypothesized  that 
differences  between  Navy  and  Marine  Corps  recruiters  might  warrant  a  sep¬ 
arate  selection  device.  The  authors  elected  to  develop  a  Marine  Corps 
Recruiter  Interest  Scale  (MCRIS)  key  based  on  Marine  Corps  recruiter  re¬ 
sponses  and  administered  the  SVIB  to  a  second  sample  of  Marine  Corps 
recruiters  (N  *  91)  attending  the  Marine  Corps  recruiting  school.  Two 
years  after  the  SVIB  administration  performance  ratings  for  these 
recruiters  were  collected  from  supervisors.  This  criterion  measure  of 


recruiter  performance  was  used  to  dichotomize  the  combined  sample  (N  =  168, 
consisting  of  77  recruiters  from  the  original  sample  and  91  Marine  Corps 
students  at  the  recruiter  school)  into  below-average  and  above-average 
recruiters.  SVIB  items  were  then  weighted  according  to  the  differential 
endorsement  patterns  of  the  two  groups,  and  included  in  the  MCRIS  if 
endorsement  differences  between  above-  and  below-average  recruiters 
exceeded  15  percent.  This  procedure  resulted  in  75  items  for  the  key. 
(Interestingly,  only  13  of  the  75  items  in  MCRIS  are  also  found  in  the 
RIS.) 

Responses  from  both  of  Graf  and  Bower’s  Marine  Corps  recruiter  samples 
were  scored  together  using  the  RIS  and  the  MCRIS,  resulting  in  validity 
coefficients  of  .67  for  the  MCRIS  and  .29  for  the  RIS.  These  two  coeffi¬ 
cients  cannot  be  compared  directly,  however,  because  the  RIS  validity 
coefficient  reflects  the  key’s  cross-validity,  while  the  cross-validity  of 
the  MCRIS  cannot  be  gauged  from  these  data.  Graf  and  Bower  recommended 
assessing  the  MCRIS’s  utility  as  a  selection  device  by  evaluating  the 
cross-validity  of  the  key  on  a  larger,  more  representative  sample  of  Marine 
Corps  recruiters.  They  also  indicated  concern  for  their  criterion  measure 
of  recruiter  performance  and  recommended  attempting  to  develop  a  more 
reliable  method  of  measuring  recruiter  performance. 

The  concerns  of  Graf  and  Bower  (1976)  mentioned  above  are  similar  to 
those  of  Abrahams  et  al .  (1973)  and  point  to  some  important  criterion- 
related  issues.  Ideally,  recruiter  performance  criteria  should  be  un¬ 
contaminated  by  extraneous  factors,  should  show  consistent  agreement  by 
multiple  raters  on  the  performance  of  individuals,  and  should  cover  all 
aspects  of  recruiter  performance.  Although  commanding  officer  nominations 
have  proven  useful,  such  single  overall  judgments  of  effectiveness/ineffec¬ 
tiveness  cannot  reflect  individual  differences  in  different  aspects  of 
performance.  Thus,  multiple  facets  of  the  recruiter  job  cannot  be  as¬ 
sessed.  Additionally,  officers  may  rely  upon  the  reputations  of  their 
subordinate  recruiters  in  making  nominations  or  ratings  instead  of  actual 
observation  of  recruiter  performance. 

Borman,  Hough,  and  Dunnette  (1976).  Explicitly  defining  the  behaviors 
leading  to  effective  or  ineffective  performance  is  one  approach  that  may 
focus  raters  on  actual  performance.  Borman,  Hough,  and  Dunnette  (1976) 
employed  such  a  strategy  in  developing  behavior  based  rating  scales  for 
Navy  recruiters.  These  authors  used  the  behavior  analysis  methodology 
(Flanagan,  1954;  Smith  &  Kendall,  1963)  to  develop  eight  9-point  behav- 
iorally  anchored  rating  scales:  Locating  and  Contacting  Qualified  Pros¬ 
pects;  Gaining  and  Maintaining  Rapport;  Obtaining  Information  from  Pros¬ 
pects  and  Making  Good  Navy-Person  Fits;  Salesmanship  Skills;  Establishing 
and  Maintaining  Good  Relationships  in  the  Community;  Providing  Knowledge¬ 
able  and  Accurate  Information  about  the  Navy;  Supporting  Other  Recruiters 
and  the  Command;  and  Administrative  Skills.  These  scales,  along  with  nine 
unanchored  scales  (including  an  Overall  Performance  Scale),  were  used  to 
gather  performance  ratings  employed  as  criteria  in  validation  studies 
conducted  by  Borman  and  his  colleagues  (Borman,  Rosse,  &  Abrahams,  1980; 
Borman,  Rosse,  &  Rose,  1983;  Borman,  Toquam,  and  Rosse,  1979). 

Borman.  Toauam.  and  Rosse  (19791.  In  this  study,  the  authors  developed 
an  empirically  keyed  non-cognitive  selection  battery  called  the  Special 
Assignment  Battery  (SAB)  for  enlisted  Navy  and  Marine  Corps  recruiters. 
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Their  research  entailed  five  major  steps:  (1)  criterion  measures  (rating 
scales  and  production  measures)  were  developed;  (2)  ratings  data  were  col¬ 
lected  and  analyzed;  (3)  a  predictor  battery  was  developed  on  the  basis  of 
hypothesized  predictor-criterion  relationships  and  was  administered  to 
recruiters;  (4)  data  collected  on  the  preliminary  battery  were  used  to 
develop  new  item  pools  for  a  second  validation  step;  and  (5)  the  final 
battery  consisting  of  personality,  vocational  interest,  and  background 
items  was  employed  in  a  concurrent  validation  study. 

After  developing  Navy  recruiter  rating  scales,  Borman  and  his  collea¬ 
gues  developed  a  revised  rating  version  for  Marine  Corps  recruiters.  They 
collected  peer,  self,  and  supervisor  ratings  for  267  Navy  recruiters  and 
118  Marine  Corps  recruiters  and  pooled  the  self-peer-supervisor  ratings. 

The  median  interrater  reliability  was  .57  for  the  pooled  Navy  recruiter 
ratings  and  was  .48  for  the  pooled  Marine  Corps  sample.  A  factor  analysis 
of  the  ratings  was  then  performed  for  the  combined  Navy/Marine  Corps  sample 
yielding  a  three  factor  solution:  Selling  Skills,  Organizing  Skills,  and 
Human  Relations  Skills.  A  fourth  criterion,  Overall  Performance,  was  added 
to  serve  as  a  summary  criterion. 

Borman  et  al .  also  gathered  production  data  (the  next  section  will 
discuss  problems  and  limitations  of  production  as  a  criterion)  for  each 
recruiter.  This  was  defined  as  the  number  of  accessions  over  the  six-month 
period  from  May  to  October  1976.  Recruiters’  production  data  were  stan¬ 
dardized  within  recruiting  district  to  provide  a  production  index  adjusted 
for  geographic  location.  This  production  index,  the  scores  on  the  three 
factors  from  the  ratings  data,  and  Overall  Performance  provided  five 
criteria. 

Intercorrelations  among  the  five  criteria  yielded  similar  results  for 
the  Navy  and  Marine  Corps  samples.  Overall  Performance  correlated  with  the 
Selling  Skills,  Human  Relations  Skills,  and  Organizing  Skills  crite¬ 
ria  .79,  .50,  and  .51  for  the  Navy  and  .72,  .42,  and  .40  for  the  Marine 
Corps  sample.  The  production  index  correlated  .52  (Navy)  and  .59  (Marine 
Corps)  with  Overall  Performance  and  .43  (Navy)  and  .45  (Marine  Corps)  with 
Selling  Skills.  The  remaining  correlations  were  .30  or  below. 


Borman  et  al .  (1979)  had  selected  non-cognitive  items  they  hypothesized 
would  predict  recruiter  performance  in  each  of  four  performance  categor- 
ies--Overall  Performance,  Selling  Skills,  Organizing  Skills,  and  Human 
Relations  Skills.  The  selection  of  items  was  from  a  number  of  published 
instruments  on  the  basis  of  data  from  an  earlier  pretest  (N  -  62  Navy 
recruiters).  Pretest  intercorrelations  between  the  predictor  items  and  the 
five  criteria  had  been  examined  to  identify  pools  of  valid  personality, 
interest,  and  biographical  items  for  each  of  the  four  performance 
categories.  The  post-pretest  battery,  entitled  the  Special  Assignment 
Battery  (SAB),  contained  310  personality  items,  the  entire  325-item  Strong- 
Campbell  Interest  Inventory,  and  a  Biographical  Survey.  The  authors  also 
developed  12  keys  for  the  SAB  with  four  pools  of  personality  items  (one  for 
each  performance  category),  four  pools  of  interest  items,  and  four  pools  of 
biographical  items. 
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The  SAB  was  then  administered  to  the  267  Navy  and  118  Marine  Corps 
recruiters  for  whom  rating  data  were  concurrently  obtained.  Correlations 
between  the  predictor  keys  and  the  five  criteria  were  examined  and  cross- 
validated  via  a  Monte  Carlo  procedure  (separately  for  the  Navy  and  Marine 
Corps  samples). 

For  both  the  Navy  and  Marine  Corps  samples,  personality  and  vocational 
interest  keyed  scores  were  significantly  correlated  with  the  four  perfor¬ 
mance  criteria  they  were  targeted  to  predict.  These  scores  were  not  signi¬ 
ficantly  related,  however,  to  the  production  index.  The  biographical 
survey  keys  produced  low  validities  for  both  types  of  criteria. 

Data  from  the  Navy  sample  (N  -  267)  alone  were  then  used  to  develop  new 
composite  SAB  keys  that  were  again  cross-validated  via  the  Monte  Carlo 
procedure.  This  resulted  in  four  composite  SAB  keys  that  combined  per¬ 
sonality,  interest,  and  biographical  items  for  predicting  each  of  the 
performance  criteria.  The  validities  of  these  final  composites  against  the 
four  performance  criteria  are  shown  in  Table  2. 

Borman.  Rosse,  and  Abrahams  (19801.  In  finalizing  and  revising  the 
SAB,  these  authors  used  the  data  described  above  to  develop  new  items  and 
cross-val idate  old  items  for  the  SAB  and  to  investigate  the  personal  char¬ 
acteristics  tapped  by  each  key.  Their  first  step  was  to  factor  analyze 
(Navy  sample,  N  *  267)  the  items  within  the  eight  item  pools  (i.e.,  four 
sets  of  personality  items  and  four  sets  of  interest  items)  to  identify 
underlying  personality  and  interest  constructs.  The  constructs  identified 
in  this  analysis  appear  in  Table  3.  Next,  they  wrote  new  items  targeted 
toward  tapping  these  constructs.  These  new  items,  along  with  items  found 
previously  (Borman  et  al . ,  1979)  to  be  valid,  formed  a  new  SAB  which 
included  four  keys  for  personality  items  and  four  keys  for  interest  items 
(i.e.,  one  for  each  performance  construct). 


^Most  methods  for  estimating  the  cross-validity  of  predictors  involve  four 
steps:  (1)  dividing  the  total  sample  into  two  subsamples  at  random  (a 
developmental  sample  and  a  cross-validation  sample);  (2)  developing  a  pre¬ 
diction  equation  using  the  developmental  sample;  (3)  using  the  interim 
prediction  function  to  compute  point  predictions  for  each  subject  in  the 
cross-validation  sample;  and  (4)  computing  the  appropriate  measure  of 
association  that  is  considered  the  estimate  of  validity  (usually  a  Pears- 
anian  coefficient).  Because  the  validity  estimated  in  this  manner  is 
conditional  upon  a  particular  "split"  or  randomization  of  the  total  sample, 
Borman  et  al .  (1979)  employed  a  Monte  Carlo  technique  that  repeats  steps  1- 
4  above,  randomizing  samples  and  reestimating  the  prediction  function  with 
each  repetition.  The  standard  error  of  the  mean  within-sample  distribution 
was  computed  with  each  repetition.  When  the  standard  error  was  suffi¬ 
ciently  reduced,  repetitions  were  stopped.  This  process  gave  rise  to  a 
distribution  of  "cross-validities,"  the  mean  of  which  was  taken  as  the 
estimate  of  validity. 


Table  2 


SAB  Composites/Keys 

(Pooled)  Validity  Estimates 

Performance 

Criteria 

Selling 

Skills 

Human 

Relations 

Skills 

Organizing 

Skills 

Overal 1 
Performance 

Navy1  .24** 

.17** 

.31** 

.22** 

Marine  Corpsz  .22* 

.22* 

.38** 

.27** 

Jn  -  267 
*  jx.05 

**  fi<.01 

i 

i 

The  new  SAB  was  then  administered  to  a  new  sample  of  194  Navy  enlisted 
recruiters.  Again,  peer  and  supervisory  ratings  and  production  data  were 
gathered  as  criteria.  Ratings  were  obtained  for  17  rating  dimensions 
including  eight  Behavior  Summary  Scales,  Overall  Performance,  and  eight 
other  scales.  Interrater  reliabilities  for  the  pooled  peer  and  supervisor 
ratings  ranged  from  .34  to  .77  with  a  median  of  .51.  A  factor  analysis  of 
the  pooled  ratings  yielded  a  three  factor  solution--Sell ing  Skills,  Human 
Relations  Skills,  and  Organizing  Ski 11s- -which  was  very  similar  to  the 
solutions  generated  previously  (Borman  et  al.,  1979). 

Because  new  items  targeted  toward  tapping  specific  constructs  were 
included  on  the  SAB,  Borman  et  al .  first  computed  intercorrelations  between 
old  and  new  predictor  construct  composites  of  items  to  ensure  that  the 
items  were  indeed  measuring  the  constructs  they  were  intended  to  measure. 
For  personality  items,  the  median  correlation  was  .56  (jk.OOI),  and  for 
interest  items  the  median  correlation  was  .67  ({K.OOl),  indicating  that  the 
new  items  did,  in  fact,  measure  the  target  constructs. 

The  authors  then  computed  validities  of  the  old  item  composites,  the 
new  item  composites,  and  composites  of  old-plus-new  items  against  the 
supervisor/peer  ratings  criteria.  For  each  performance  category  (e.g., 
Selling  Skills)  a  single  SAB  composite  was  formed  across  the  personality 
items  targeted  toward  that  criterion,  and  another  composite  was  formed  for 
the  interest  items.  The  old-plus-new  personality  item  composites  corre¬ 
lated  .23  (fic.01)  with  Selling  Skills,  .24  (jk.01)  with  Human  Relations 
Skills,  .15  (e<.05)  with  Organizing  Skills,  and  .22  (jk.01)  with  Overall 
Performance.  Similarly,  the  old-plus-new  interest  item  composites  corre¬ 
lated  significantly  (fi<.01)  with  Selling  Skills  (r  =  .21),  Human  Relations 
Skills  (r  -  .22),  and  Overall  Performance  (r  =  .22). 
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Table  3 


Factors  Identified  for  Each  Performance  Category 


Performance 

Categories 

Factor/Construct 

Personality  Items 

Selling  Skills 


5 


1.  Good  impression. 

2.  Impulsive,  carefree  vs.  order,  planning  ahead,  system¬ 
atic,  level-headed. 

3.  Enjoying  being  center  of  attention,  leading,  showing 
off,  and  speaking  before  a  group. 

a.  Working  hard  and  with  confidence,  being  happy  vs. 
being  unhappy,  giving  up  easily,  disgruntled  about  life. 


Human  Relations 
Skills 


1.  Preference  for  working  with  and  being  with  people. 

2.  Spontaneity,  impulsivity,  "fast  and  careless,"  rebel' 
lious,  tendency  to  have  bad  moods. 

3.  Unhappy,  lack  of  confidence,  disgruntled  about  life. 

4.  Ambitious,  working  hard,  pushing  self. 


Organizing  Skills  1.  Order,  planning  ahead,  well  organized  vs  impulsive, 

acting  without  thinking,  "fast  and  careless." 

2.  Leading  and  influencing  others,  giving  orders,  demand¬ 
ing  of  self,  ambitious,  dominant. 

3.  Unhappy,  discouraged,  doing  little  in  life,  giving  up 
hope,  f  eeling  useless. 

4.  "Bad  actor,"  was  unruly  and  rebellious  in  school, 
unsocialized. 


Overall 

Performance 


I 


1.  Doing  more  than  expected  vs.  giving  up,  working  just 
hard  enough. 

2.  Impulsive,  "fast  and  careless"  vs.  order,  methodo¬ 
logical,  planning  ahead. 

3.  Leading  and  influencing  others,  dominant,  strong  per¬ 
sonality. 

a.  Good  impression  vs.  admitting  occasional  meanness, 
grouchiness,  disgust  with  self,  discouragement,  use¬ 
lessness,  bad  mood. 

3.  People  oriented,  liking  to  be  around  others  and  close 
to  others,  open  to  other  people. 


I 

I 


Vocational  Interest  Items 


Selling  Skills 


Human 

Relations 

Skills 


Organizing 

Skills 


1.  Interest  in  extroverted,  dominant,  leadership  activities 
and  occupations. 

2.  Interest  in  occupations  involving  attention  to  detail. 

3.  Interest  in  law  and  politics. 

a.  Interest  in  sports  and  competitive  activities. 

1.  Interest  in  dominant,  extroverted,  social  activities. 

2.  Interest  in  teaching  and  counseling. 

3.  Interest  in  "feminine"  occupations  and  activities. 

а.  Interest  in  newspaper  reporting  and  foreign  service. 

5.  Interest  in  sports  and  competitive  activities. 

б.  Interest  in  religion  and  in  being  around  the  sickly. 

1.  Interest  in  politics  and  high  level  management  jobs. 

2.  Interest  in  bookkeeping,  statistical,  and  detail  work. 

3.  Interest  in  "feminine"  occupations  and  activities.1 2 3 
a.  Interest  in  leadership  and  responsibility. 


Overall 

Performance 


1.  Interest  in  law  and  poltics,  and  management  occupa¬ 
tions  and  activities. 

2.  Interest  in  activities  and  occupations  that  require 
extroversion,  dominance,  responsibility,  and  leader¬ 
ship. 

3.  Interest  in  sports  and  competitive  activities. 

a.  Interest  in  teaching  and  counseling. 

J.  Interest  in  "feminine"  occupations.3 


aThese  constructs  related  negatively  to  their  target  performance  criteria. 
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To  develop  a  final  SAB,  the  authors  examined  the  cross  validities  for 
the  old  SAB  personality  and  interest  items  and  selected  items  that  demon¬ 
strated  consistent  validity  in  both  samples  (N  =  267  and  N  =  194).  New 
items  demonstrating  good  validity  in  the  second  sample  (N  =  194)  were  also 
included  in  the  final  SAB.  One  hundred  and  ten  personality  and  sixty 
interest  items  formed  four  keys  (e.g.,  one  key  for  Selling  Skills  which 
included  personality  and  interest  items,  one  key  for  Human  Relations  Skills 
which  included  personality  and  interest  items,  etc.).  Correlations  between 
the  final  SAB  scores  and  pooled  supervisory  and  peer  ratings  for  the  second 
sample  (N  «  194)  were  .43  with  Selling  Skills,  .46  with  Human  Relations 
Skills,  .40  with  Organizing  Skills,  and  .43  with  Overall  Performance.  The 
SAB  scores  were  also  correlated  with  each  recruiter’s  raw  production  data. 
These  validities  were  .22,  .23,  .13,  and  .26  respectively;  three  of  which 
were  significant  (jk.OI). 

In  summary,  results  from  research  by  Borman  and  his  colleagues  have 
shown  that  non-cogniti ve  variables  are  useful  predictors  of  both  recruiter 
performance  (supervisor  ratings)  and  effectiveness  (production).  In  addi¬ 
tion,  two  aspects  of  this  work  warrant  special  attention.  First,  ratings 
on  the  17  scales  used  in  this  research  were  factor-analyzed  using  three 
separate  samples,  and  all  analyses  yielded  similar  three-factor  solutions. 
Thus,  it  appears  that  the  ratings  obtained  in  these  studies  were  tapping 
three  relatively  stable  constructs  underlying  Navy  recruiter  performance. 
Second,  although  several  other  authors  have  developed  empirically  keyed 
predictors  (e.g.,  Abrahams  et  al.,  1973),  they  have  not  investigated  the 
individual  differences  constructs  being  tapped  by  such  predictors.  In 
contrast,  the  Borman  work  represents  a  mix  of  both  empirical  and  construct 
validation.  (Because  the  SAB  has  been  employed  in  several  studies,  we  have 
briefly  summarized  relevant  research  in  Appendix  A.) 

Borman.  Rosse,  and  Rose  (19831.  In  an  effort  to  predict  Navy  officer 
recruiter  performance,  the  SAB  was  again  employed.  Borman,  Rosse,  and  Rose 
(1983)  examined  the  SAB  items,  evaluated  the  appropriateness  of  the  item 
content  for  predicting  officer  recruiter  performance,  and  wrote  new  items 
targeted  toward  constructs  identified  in  the  earlier  research.  Items  from 
the  SAB  keys  developed  for  enlisted  recruiters  (i.e.,  Selling  Skills,  Human 
Relations  Skills,  Organizing  Skills,  and  Overall  Performance)  were  used  to 
form  four  corresponding  officer  recruiter  composites.  These  composites 
were  then  correlated  with  supervisor  ratings  to  assess  the  validity  of 
these  measures  for  predicting  Navy  officer  recruiter  success. 

The  predictor  battery  was  administered  to  132  commissioned  officer 
recruiters  with  three  months  or  more  recruiting  experience.  Self  and 
supervisory  ratings  were  also  gathered  for  each  of  the  132  recruiters  on  9- 
point  behaviorally  anchored  rating  scales  and  an  Overall  Performance  scale. 
The  median  intraclass  correlation  between  supervisor  and  self-ratings  on 
the  behavior  summary  scales  was  .36,  and  on  the  overall  performance  dimen¬ 
sion  the  intraclass  correlation  was  .49. 


^Navy  officer  recruiters  are  commissioned  officers  whose  job  involves 
recruiting  officers. 
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Validity  estimates  were  computed  separately  for  the  personality  and 
vocational  interest  scales.  Three  of  the  four  personality  keys--Human 
Relations  Skills,  Administrative  Skills  and  Overall  Performance--correlated 
significantly  with  the  performance  criteria  (.30,  .18,  and  .19  respec¬ 
tively).  Also,  two  of  the  interest  keys--Sell ing  Skills  and  Overall  Per¬ 
formance-  -correl  ated  significantly  with  the  criteria  (both  r’s  =.16). 

The  authors  interpret  the  results  of  their  work  with  Navy  officer 
recruiters  as  indicating  that: 

"successful  officer  recruiters  tend  to  be  hardworking  and  ambitious, 
and  they  push  themselves  hard.  They  are  also  strongly  oriented  toward 
people  activities.  They  like  to  be  close  to  others,  and  they  can  be 
spontaneous  and  fun-loving  in  these  social  activities.  Related  to 
vocational  interests,  effective  officer  recruiters  are  especially  in¬ 
terested  in  jobs  and  activities  that  require  extraversion,  dominance, 
responsibility,  and  leadership  (p.  10)." 

Reviewing  the  results  of  their  work  across  studies,  Borman  and  his 
colleagues  note  that  there  are  similarities  and  differences  between  pat¬ 
terns  of  validities  for  officer  and  enlisted  recruiters.  In  the  per¬ 
sonality  realm,  the  hard  work,  and  high  ambition  theme  is  present  in  both 
groups  of  successful  recruiters.  Additionally,  successful  enlisted  and 
officer  recruiters  are  characterized  by  spontaneity,  even  some  of  the 
negative  aspects  of  it  such  as  liking  a  "fast  and  careless"  lifestyle  and 
having  a  tendency  to  be  a  bit  rebellious.  Effective  officer  recruiters  are 
more  likely  to  have  a  strong  people  orientation,  including  a  liking  to  be 
with  and  be  close  to  others.  Conversely,  successful  enlisted  recruiters  are 
characterized  by  an  enjoyment  of  being  the  center  of  attention,  even  of 
"showing  off".  The  more  successful  officer  recruiters  apparently  project  a 
more  subtle  people  orientation. 

Regarding  vocational  interests,  a  strong  orientation  toward  social 
activities  and  toward  occupations  that  require  an  outgoing  and  dominant 
self-presentation  is  characteristic  of  both  officer  and  enlisted  recruit¬ 
ers.  Interests  that  describe  successful  enlisted  recruiting  personnel,  but 
not  so  much  effective  officer  recruiters,  relate  to  sports  and  competitive 
activities  and  toward  the  law  and  politics.  Successful  officers  have 
greater  orientation  toward  leadership  and  responsibility  than  do  successful 
enlisted  recruiters. 

In  summary,  a  number  of  recruiter  selection  studies  employed  supervisor 
ratings  or  commanding  officer  nominations  as  criteria  for  assessing  the 
validity  of  predictor  measures.  Non-cognitive  predictors  have  proven 
successful  in  several  studies.  The  work  of  Abrahams  et  al .  (1973),  Borman 
et  al .  (1979),  Graf  and  Bower  (1976),  and  Wollack  and  Kipnis  (I960) 
suggests  that  vocational  interests  may  successfully  predict  military 
recruiter  performance.  Studies  employing  personality  measures  as  predic¬ 
tors  (cf.  Borman  et  al . ,  1979;  Krug,  1972)  found  reasonably  high  relation¬ 
ships  between  these  predictors  and  recruiter  performance.  Background  or 
biographical  measures  have  also  predicted  recruiter  performance  with  some 
success  (Borman  et  al.,  1979;  Krug,  1972).  Considering  all  the  work  we 
have  reviewed,  a  significant  relationship  between  non-cognitive  predictors 
and  recruiter  performance  appears  to  be  a  consistent  finding. 


Cognitive  predictors,  on  the  other  hand,  have  met  with  little  success 
in  predicting  recruiter  performance.  Intelligence  and  aptitude  measures 
have  shown  virtually  no  relationship  with  recruiter  performance.  Only 
verbal  measures  have  yielded  even  moderate  relationships  with  performance 
criteria  (cf.  Brown  et  al.,  1975;  Wollack  &  Kipnis,  1969).  Overall,  the 
cognitive  predictor-recruiter  performance  (i.e.,  measured  by  ratings  or 
nominations)  relationship  appears  to  be  low  and  of  little  utility  for 
recruiter  selection  research. 
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Productivity  as  a  Measure  of  Recruiter  Effectiveness 


Because  recruiter  production  {i.e.,  gross  number  of  recruits  over  a 
defined  time  period)  is  essentially  the  "bottom-line"  criterion  in 
recruiting  and  is  the  primary  factor  used  by  the  armed  services  to  evaluate 
recruiter  success,  identifying  variables  that  might  predict  recruiter 
productivity  is  of  particular  importance.  Unfortunately,  the  usefulness  of 
production  criteria  in  selection  research  is  limited  by  two  problems. 

First,  objective  production  data  are  sensitive  to  environmental  factors 
beyond  the  recruiter’s  control.  Second,  production  data  probably  fail  to 
tap  all  facets  of  the  recruiter  job.  For  instance,  production  data  may  not 
reflect  recruiters’  efforts  to  maintain  rapport  with  the  community  or  to 
provide  support  to  other  recruiters. 

Research  employing  production  criteria  has  proceeded  in  two  areas: 

(1)  research  using  production  levels  as  criteria  for  validating  predictors; 
and  (2)  research  investigating  environmental  contaminants  of  production. 

We  have,  therefore,  divided  this  chapter  into  two  sections.  In  the  first 
section  we  summarize  research  employing  production  as  a  primary  criterion 
for  predictor  validation,  and  we  discuss  research  focusing  on  factors 
influencing  production  in  the  latter  section. 

Production  as  Criteria  for  Validating  Predictors 


Broqden  and  Taylor  (1949).  In  an  attempt  to  predict  Army  recruiter 
effectiveness,  Brogden  and  Taylor  (1949)  developed  a  battery  consisting  of 
four  paper-and-pencil  predictors.  These  predictors  included  a  measure  of 
vocational  interests  and  three  measures  tapping  interests,  hobbies,  and 
background.  The  four  predictor  scales  were  then  item-analyzed  against  a 
production  criterion  consisting  of  the  average  number  of  recruits  brought 
into  the  Army  by  each  recruiter  per  hour  on  recruiting  duty.  The  resulting 
empirically  keyed  predictors  were  cross-validated  using  a  sample  of  475 
recruiters,  yielding  a  validity  coefficient  of  .18. 

Brogden  and  Taylor  postulated  that  this  low  validity  was  due  to  the 
unreliability  of  the  production  criterion  and  to  the  contamination  poten¬ 
tially  present  in  such  global  objective  indices.  The  authors  reanalyzed 
the  data  using  a  turnover  criterion.  When  this  criterion  was  used,  the 
summed  scores  of  the  four  inventories  yielded  a  biserial  validity  coef¬ 
ficient  of  .36. 

The  focus  of  the  Brogden  and  Taylor  study  was  on  the  criterion  employed 
to  validate  predictors  for  use  in  selecting  Army  recruiters.  The  initial 
production  criterion  was  criticized  because  it  was  thought  to  contain  many 
factors  not  associated  with  recruiter  effectiveness.  Of  course,  the  turn¬ 
over  criterion  may  also  reflect  factors  unrelated  to  recruiter  effective¬ 
ness.  The  turnover  criterion  did,  however,  result  in  a  higher  validity 
when  it  was  correlated  with  the  predictors.  This  increase  in  validity  may 
be  due  in  part  to  the  relatively  low  reliability  and  significant  contamina¬ 
tion  associated  with  the  production  criterion.  It  is  also  possible  that 
measures  such  as  those  used  here  are  simply  better  predictors  of  turnover 
than  they  are  of  recruiter  production. 
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Larriva  (1975).  In  an  effort  to  overcome  problems  related  to  the  gross 
production  criterion,  Larriva  developed  an  adjusted  production  index  for 
use  in  a  concurrent  validity  study.  All  recruiters  from  a  single  Marine 
Corps  District  (N  =  470)  were  administered  the  predictor  battery  which 
included: 

1.  The  16PF--a  personality  test  designed  to  measure  an  individual’s  per¬ 
sonality  in  terms  of  16  basic  factors. 

2.  A  25-item  experimental  supplement  designed  to  measure  motivational 
distortion  (a  lie  scale)  and  the  strength  of  motivation  to  succeed  as  a 
recruiter. 

3.  Seven  biographical  items  yielding  information  about  years  of  service, 
age,  sex,  marital  status,  number  of  dependents,  years  of  formal  educa¬ 
tion,  and  population  of  subject’s  home  of  record. 

The  Recruiting  Command  made  arrangements  to  record  the  number  of  Non- 
Prior  Service  (NPS)  accessions  for  each  recruiter  throughout  the  year  1974. 
This  was  the  initial  production  criterion  used  in  the  research. 

After  all  the  completed  inventories  were  returned  and  the  first  quarter 
accession  data  were  reported,  Larriva  analyzed  the  data  using  a  regression 
formula  developed  by  the  Navy  to  predict  recruiter  performance  (cf.  Krug, 
1972).  This  analysis  indicated  that  the  Navy  formula  was  not  a  valid 
equation  for  predicting  gross  Marine  Corps  recruiter  production.  Larriva 
conjectured  that  the  accessions  criterion  might  not  provide  relevant  effec¬ 
tiveness  information;  thus,  he  generated  several  effectiveness  indices  on 
the  basis  of  assumptions  about  and  corrections  for  the  recruiting  en¬ 
vironment  and  geographical  differences  in  production. 

Using  first  and  second  quarter  accession  data,  each  production  index 
was  employed  as  the  criterion  measure  and  correlated  with  the  personality, 
demographic,  and  experimental  variables  of  the  predictor  inventory. 

Larriva  then  examined  the  resulting  predictor-criterion  relationships  and 
selected  the  criterion  index  that  yielded  the  most  valid  multiple  correla¬ 
tion  coefficient.  The  index  selected,  referred  to  as  P16,  indicated  the 
number  of  accessions  for  urban  and  rural  recruiters  separately  and 
corrected  for  geographical  differences  in  relative  effectiveness  of 
recruiters.  Accordingly,  the  validation  sample  was  split  into  an  Urban 
Recruiter  Group  (N  =  308)  and  a  Rural  Recruiter  Group  (N  =  162),  and  a 
weight  correcting  for  geographical  differences  was  applied  to  production 
scores.  Larriva  also  developed  subsets  of  these  two  samples  of  urban  and 
rural  recruiters  that  contained  only  the  best  and  poorest  recruiters. 

These  two  reduced  samples  were  referred  to  as  the  Urban  Hi -Low  (N  =  122) 
and  the  Rural  Hi -Low  (N  =  96)  groups.  The  P16  criterion  was  then  used  as 
the  effectiveness  measure  (criterion)  and  stepwise  multiple  regression  was 
utilized  to  generate  four  formulae  for  predicting  recruiter  success  from 
the  inventory  data  (i.e.,  one  for  each  of  the  two  full  samples--Urban  and 
Rural,  and  one  for  each  of  the  subset  samples--Urban  Hi -Low  and  Rural  Hi- 
Low) . 


The  means  and  standard  deviations  of  the  predicted  and  actual  scores 
were  calculated  for  the  four  data  bases  and  were  used  to  establish  cutoff 


scores.  These  cutoff  scores  were  then  used  to  assess  the  effectiveness  of 
the  prediction  equations  and  to  select  or  delete  recruiters  in  a  cross- 
validation  sample.  The  cross-validation  sample  consisted  of  98  recruiters 
who  were  not  included  in  the  validation  sample  but  who  had  a  minimum  of 
three  months  accession  data.  As  prescribed  by  the  production  index,  the 
sample  was  split  into  an  Urban  Recruiter  Group  (N  -  66)  and  a  Rural  Re¬ 
cruiter  Group  (N  =  32),  effectiveness  scores  (the  P16)  were  generated  for 
each  recruiter,  and  subsets  of  the  two  full  samples  (i.e.,  Urban  and  Rural) 
were  again  developed. 

The  two  urban  prediction  equations  were  applied  to  the  urban  cross- 
validation  sample  and  results  of  the  prediction  equations  were  compared 
according  to  accuracy  and  expected  increase  in  productivity.  The  same 
procedures  were  followed  to  compare  the  results  of  the  Rural  prediction 
equation  and  Rural  Hi -Low  prediction  equation  when  applied  to  the  rural 
cross-validation  sample.  Results  for  the  two  comparisons  are  listed  in 
Table  4. 

Table  4 

Comparison  of  Rural  and  Urban  Recruiter  Samples 
After  Application  of  Regression  Equations 


Urban 

Urban 

Hi -Low 

Rural 

Rural 
Hi -Low 

Number  deleted  by 
cutting  score 

24 

34 

12 

12 

Percent  correct 
cutoff 

88% 

76% 

75% 

83% 

Mean  production  score 
of  those  deleted 

9.29 

9.91 

10.06 

9.02 

Mean  production  score 
of  those  remaining 

12.29 

12.54 

12.39 

13.02 

Expected  increase  in 
productivity 

10% 

12% 

7.6% 

13% 

From  these  comparisons,  Larriva  concluded  that  the  Urban  predictor 
equation  and  Rural  Hi -Low  equation  would  prove  successful  in  selecting 
Marine  Corps  recruiters.  The  author  also  suggested  that  both  equations 
could  be  used  to  analyze  each  predictor  inventory,  thus  allowing  the  Marine 
Corps  Recruiting  Command  to  select  the  optimal  setting,  urban  or  rural,  for 
each  new  recruiter. 

In  summary,  when  recruiting  setting  (urban-rural)  and  geographical 
differences  were  accounted  for  in  the  criterion,  relatively  successful 
prediction  equations  were  formed  for  urban  recruiters  and  for  rural  re- 


cruiters.  Cross-validity  results  suggest  that  the  predictor  inventory  may 
be  a  useful  selection  device  for  screening  candidates  for  the  Marine  Corps 
recruiter  job.  The  Navy  prediction  equation  did  not  effectively  predict 
unadjusted  Marine  Corps  recruiter  production.  The  original  Navy  equation 
was  developed,  however,  against  commanding  officer  nomination  criteria 
rather  than  against  production.  Thus,  the  low  cross-validity  may  indicate 
that  the  Navy  equation  is  more  useful  in  predicting  performance  criteria 
than  production.  It  would  be  of  interest  to  apply  the  Navy  prediction 
equation  for  the  Larriva  inventory  to  the  cross-validation  sample  using  the 
P16  criterion.  Apparently,  Larriva  never  correlated  the  Navy  prediction 
equation  scores  with  scores  on  the  P16  criterion. 

A  final  point  of  interest  in  this  research  has  to  do  with  the  selection 
of  a  criterion.  Larriva  essentially  selected  the  criterion  that  correlated 
most  highly  with  the  predictors.  A  clearly  more  acceptable  (and  justi¬ 
fiable)  method  is  to  define  the  criterion,  or  criteria,  as  precisely  as 
possible  and  then  to  select  a  measure  that  provides  relevant  and  reliable 
measurement  of  each  criterion  without  regard  to  the  predictors.  The  cross- 
validation  procedure  employed  in  Larriva’s  study  makes  this  criticism  less 
severe,  but  the  P16  criterion  would  have  been  better  justified  as  a  measure 
of  effectiveness  if  predictor  data  had  not  dictated  its  choice. 

Atwater.  Abrahams,  and  Trent  (1986).  Recently,  Atwater  et  al .  (1986) 
have  conducted  two  studies  employing  production  data  as  criteria  for 
validating  the  Special  Assignment  Battery  (SAB)  (cf.  Borman  et  al.,  1983): 
(1)  a  large-scale  concurrent  validation  study;  and  (2)  a  predictive 
validation  study. 

For  the  concurrent  study,  these  authors  administered  the  SAB  to  1,005 
Marine  Corps  recruiters  who  had  been  on  the  job  for  at  least  six  months. 

Two  criterion  measures  were  obtained  for  each  recruiter  -  gross  producti¬ 
vity  (average  number  of  monthly  contracts  written)  and  a  supervisor  rating 
on  a  three-point  overall  performance  scale. 

Recall  that  the  SAB  is  a  non-cogniti ve  predictor  battery  which  includes 
four  scales  -  Selling  Skills,  Human  Relations  Skills,  Organizing  Skills, 
and  Overall  Performance.  Correlations  between  the  predictor  scales  and 
gross  productivity  were  .23  (jx.01),  .15  (pc. 01),  .09  (jx.05),  and  .24 
(jk.OI),  respectively.  Correlations  between  the  predictor  scales  and  the 
overall  performance  rating  were  .19,  .15,  .13,  and  .20,  and  were  all  sig¬ 
nificant  (jk.OI).  The  authors  compared  these  results  to  those  obtained  by 
Borman  et  al .  (1983)  on  a  sample  of  Navy  recruiters  (N  =  194)  and  found 
that  the  patterns  of  correlations  were  quite  similar  and  that  there  were  no 
statistically  significant  differences  between  the  production/predictor 
correlations  obtained  in  the  two  studies. 

For  the  predictive  study,  the  SAB  was  administered  to  664  Marines  who 
were  being  screened  for  assignment  to  recruiter  duty  during  1981  and  1982 
(SAB  scores  were  not  used  in  actual  assignment  to  recruiting  duty).  Four 
hundred  and  thirty-one  of  the  664  Marines  were  assigned  to  recruiting  duty 
by  1983  and  were  tracked  through  recruiting  school  and  for  at  least  a  year 
of  recruiting  duty. 

Two  criteria  were  employed  for  validating  the  predictor  battery  - 
production  and  completion  of  tour.  The  production  index  was  net  monthly 
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production  computed  by  subtracting  the  number  of  recruits  who  left  the 
service  while  in  the  delayed  entry  program  or  during  basic  training  from 
each  recruiter’s  gross  production.  The  completion  of  tour  measure  indi¬ 
cated  whether  the  recruiter  had  completed  at  least  one  year  of  recruiting 
duty  or  had  been  removed  from  recruiting  duty. 

Of  the  431  Marines  assigned  to  recruiting  duty,  76  failed  to  complete 
recruiting  school,  and  criterion  data  were  not  available  for  40,  reducing 
the  sample  size  to  315.  Correlations  between  the  four  predictor  scales  and 
the  production  measure  were  .27  (Selling  Skills),  .17  (Human  Relations 
Skills),  .21  (Organizing  Skills),  and  .29  (Overall  Performance).  Again, 
these  correlations  were  consistent  with  those  found  previously  in  research 
with  this  predictor  battery. 

A  summary  of  the  attrition  rates  appears  in  Table  5.  The  authors 
computed  a  composite  of  the  four  SAB  scales,  the  recruiter  potential 
composite  (Recpot),  and  compared  Recpot  scores  for  the  samples  at  various 
stages  in  the  study.  As  shown  in  Table  5,  the  average  Recpot  score  for  the 
76  Marines  who  failed  to  complete  recruiting  school  was  lower  than  that  of 
those  completing  school  (N  =  355);  the  difference,  however,  was  not  statis¬ 
tically  significant.  Of  the  315  recruiters  who  were  located  after 
recruiting  school,  84  did  not  have  normal  rotation  to  their  tour  (i.e.,  73 
were  relieved  for  various  reasons,  and  11  completed  their  active  service  in 
the  Marine  Corps  before  the  year  ended).  The  mean  Recpot  score  for  these 
84  recruiters  was  nearly  one-half  standard  deviation  below  the  mean  of 
those  who  completed  their  tour.  Although  the  eleven  recruiters  who  ended 
their  active  service  were  not  dismissed  for  poor  performance,  their  produc¬ 
tion  was  significantly  below  the  average  for  Marines  who  completed  their 
tour  (1.8  recruits  per  month  vs.  2.3,  s.d.  =  1.0). 

In  sum,  the  non-cognitive  predictor  scales  (SAB)  developed  using  a 
sample  of  Navy  recruiters  successfully  predicted  Marine  Corps  recruiter 
productivity  in  a  concurrent  study  including  over  a  thousand  Marine  Corps 
recruiters  and  in  a  predictive  study  (N  =  315).  The  authors,  therefore, 
noted  that  the  similarity  of  these  results  suggests  "stability  over  time,  as 
well  as  general izabil ity  of  SAB  scores  across  services  (p.  3.)." 

Environmental  Contaminants  of  Production 


Because  environmental  factors  as  well  as  recruiters’  personal 
characteristics  (e.g.,  personality,  interests)  may  influence  productivity, 
a  number  of  studies  have  been  conducted  to  delineate  factors  outside  the 
individual’s  (recruiter’s)  control  that  might  contaminate  production  data. 
The  paragraphs  below  summarize  research  in  this  arena. 

Bennett  and  Haber  (19731.  To  investigate  factors  influencing  recruiter 
production,  these  authors  collected  data  on  several  environmental  and  back¬ 
ground  variables  from  259  Marine  Corps  recruiters.  The  recruiters  repre¬ 
sented  29  recruiting  stations  throughout  the  United  States.  The  informa¬ 
tion  was  collected  from  performance  records  and  in  a  survey  of  recruiters 
and  their  jobs.  Bennett  and  Haber  divided  the  variables  into  three  cate¬ 
gories: 
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Table  5 


|  Means,  Standard  Deviations,  and  Sample  Sizes 

i  for  the  Predictive  Recruiter  Study1 

| 


\ 


f 


Sample 

RecDOt^ 

Mean 

SD 

Of  644  applicants: 

58.5 

45.2 

431  were  assigned  to  recruiting  school 

60.4 

48.4 

213  were  not  assigned 

55.0 

47.2 

Of  431  assigned  to  recruiting  school: 

355  completed  school 

61.7 

48.3 

76  failed  to  complete 

54.4 

49.9 

Of  355  who  completed  school: 

315  were  located  in  field 

61.6 

48.1 

40  were  not  located 

62.5 

47.4 

Of  the  315  located  in  field: 

231  had  normal  tour  completion 

66.4 

44.6 

84  did  not  have  normal  tour  completion 

48.6 

53.6 

1  Atwater,  Abrahams,  &  Trent  (1986). 

^  Recruiter  potential  composite  (Recpot);  score  range:  -75  to  +193. 
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1.  Selection  variables--General  Classification  Test  scores;  age;  race; 
level  of  education;  number  of  dependents;  previous  service  as  a  career 
planner  (i.e.,  counselor)  or  drill  instructor;  method  of  assignment  to 
recruiting  duty  (volunteer/assigned);  and  opinion  about  recruiting  duty 
as  a  financial  hardship. 

2.  Deployment  variables--whether  the  recruiter  is  assigned  in  or  out  of 
his  or  her  home  state;  distance  from  home  state;  type  of  area  assigned 
to--urban/suburban  or  rural;  number  of  times  assigned;  hours  per  week 
spent  on  recruiting;  and  percentage  of  time  spent  out  of  the  office 
recruiting. 

3.  Evaluation  variab1es--number  of  months  the  individual  has  been  on  his 
or  her  current  tour  of  duty  and  the  recruiter’s  percentile  rank  in 
Marine  Corps  Recruiter  Class. 

Gross  productivity  (i.e.,  the  average  number  of  recruits  enlisted  per 
month)  was  employed  as  the  measure  of  effectiveness  for  each  recruiter;  yet 
Bennett  and  Haber  noted  that  differences  in  gross  productivity  cannot  be 
completely  attributed  to  individual  recruiter  differences  in  performance. 
Differences  may  be  due  in  part  to  a  variety  of  regional  differences  such  as 
the  number  of  qualified  prospects  or  local  attitudes  toward  the  military 
service.  For  this  reason,  the  sample  was  divided  into  two  groups  on  the 
basis  of  the  average  regional  enlistment  rates  for  1971.  The  first  group 
contained  those  recruiters  from  high  enlistment  recruiting  stations  (N  * 
109)  and  the  second  group  contained  recruiters  serving  in  low  enlistment 
areas  (N  =  150) . 

The  authors  used  multiple  regression  to  determine  the  magnitude  of  the 
relationship  between  each  variable  and  recruiter  productivity,  and  computed 
a  separate  regression  equation  for  each  enlistment  area  group.  In  the  high 
enlistment  area  group,  three  variables  were  found  to  be  significantly 
related  to  productivity.  The  first  variable,  type  of  area,  showed  that 
urban/suburban  recruiters  enlisted  more  persons  per  month  (fi<.01)  than 
rural  recruiters.  The  second  variable,  geographical  assignment,  showed 
that  recruiters  stationed  in  their  home  state  enlisted  more  persons  per 
month  (jk.01)  than  those  stationed  more  than  500  miles  away  from  their  home 
state.  The  third  indicated  that  productivity  was  negatively  related  to 
increased  reassignments  (£<.05).  These  three  variables  accounted  for  35 
percent  of  the  production  variance. 

In  the  low  enlistment  areas,  two  variables  correlated  significantly 
with  productivity.  The  first  significant  variable  indicated  that  those 
recruiters  who  feel  recruiting  duty  is  a  financial  hardship  enlist  more 
persons  per  month  (£<.05)  than  those  who  do  not.  The  second  significant 
variable  suggested  that  those  recruiters  who  had  previously  served  as 
career  planners  enlisted  a  greater  number  of  persons  per  month  (£<.05)  than 
did  those  who  had  not  been  career  planners.  These  two  variables  account 
for  only  12  percent  of  the  productivity  variance,  however,  and  it  should  be 
noted  that  the  regression  equations  obtained  in  the  study  were  not  cross- 
validated.  Thus,  the  relationships  found  here  may  well  overestimate  the 
true  usefulness  of  these  variables. 

Brown.  Mood,  and  Harris  (1975).  These  authors  also  sought  to  develop  a 
valid  criterion  measure  of  recruiter  effectiveness  by  focusing  on  an  index 
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used  unsuccessfully  in  previous  recruiter  studies--total  production  scores. 
Brown  et  al .  noted  that  these  scores  were  contaminated  by  "opportunity 
bias"  arising  from  territorial  factors  that  fall  outside  the  control  of  the 
individual  recruiter.  They  hypothesized  that  if  territorial  factors  were 
somehow  partialed  out  of  the  production  score  variance,  the  result  would  be 
a  more  realistic,  unbiased  picture  of  recruiter  effectiveness. 

Brown  and  colleagues  moved  in  this  direction  first  by  identifying 
territorial  variables  that  could  potentially  bias  production  figures.  A 
total  of  twelve  such  biasing  agents  were  noted,  including  the  average 
number  of  enlistments  per  recruiter  in  the  recruiter’s  District  Recruiting 
Command  or  DRC;  the  proportion  of  territory  that  is  metropolitan,  suburban, 
or  rural;  and  the  recruiters’  length  of  experience.  A  nationwide  random 
sample  of  Army  recruiters  was  then  developed  by  requesting  each  commander 
from  the  five  Regional  Recruiting  Commands  (RRCs)  to  supply  names  of  100 
recruiters  from  their  respective  RRCs.  Commanders  selected  recruiters 
randomly  according  to  their  social  security  numbers,  and  also  supplied 
total  production  figures  for  each  recruiter  for  the  period  from  July  to 
December  1973.  Territorial  information  from  each  DRC  represented  by  re¬ 
cruiters  in  the  sample  was  then  compiled  by  Army  personnel  from  available 
records  and  from  a  special  market  survey  they  conducted. 

Scores  on  the  twelve  territorial  factors  were  subjected  to  a  stepwise 
multiple  regression  with  total  production  scores  serving  as  the  criterion. 
Three  variables  contributed  significantly  (jx.05)  to  the  prediction  of 
production  scores:  average  production  per  recruiter  in  subject’s  DRC, 
proportion  of  all  enlistees  in  the  DRC  who  chose  the  Army  (rather  than 
another  service),  and  suburban  proportion  of  the  zone  were  positively 
related  to  production.  These  three  variables  accounted  for  51  percent  of 
the  production  score  variance. 

Next,  the  three  territorial  factors  were  used  to  develop  a  multiple 
regression  equation  that  yielded  predicted  production  scores  for  each 
recruiter.  These  predicted  scores  were  then  employed  to  compute  unbiased 
production  scores  obtained  by  computing  the  ratio  of  total  production  to 
predicted  production  scores  and  multiplying  by  100.  The  authors  termed  the 
resulting  scores  as  Benchmark  Achievement  Scores  (BAS),  which  were  cor¬ 
rected  for  DRC  production,  proportion  of  young  persons  joining  the  Army  in 
the  DRC,  and  the  proportion  of  the  DRC  that  is  suburban.  These  adjusted 
scores  should  provide  relatively  unbiased  production  information. 

Brown  et  al .  also  developed  adjusted  scores  that  are  easier  to  compute 
than  BAS;  these  scores  were  termed  Simple  Achievement  Scores  (SAS).  They 
were  derived  by  computing  a  ratio  of  a  recruiter’s  total  production  to  the 
average  production  in  the  DRC.  Because  this  ratio  was  the  variable  that 
accounted  for  most  of  the  variance  in  the  multiple  regression  equation, 
Brown  et  al .  recommended  that  the  Simple  Achievement  Scores  be  used  as 
future  criterion  measures.  Simple  Achievement  Scores  also  correlate  highly 
with  the  BAS  (r  =.96).  The  development  of  these  adjusted  production  mea¬ 
sures  was,  however,  not  complete  when  these  authors  conducted  a  validation 
study  and  they  were,  therefore,  not  employed  as  criteria. 


Arima  (1977).  Similar  to  the  Brown  et  al .  (1975)  study,  Arima  (1977) 
employed  a  regression  approach  to  identify  environmental  variables  asso¬ 
ciated  with  Navy  recruiter  productivity.  Arima  collected  data  on  several 
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educational  environment  variables  such  as  average  pay  for  local  high  school 
teachers  and  average  daily  attendance  in  local  high  schools.  He  then 
scored  268  recruiters  in  three  recruiting  districts  on  these  variables,  as 
well  as  two  dummy  variables  representing  district  membership.  The  environ¬ 
mental  variables  were  entered  into  a  step  wise  multiple  regression  analysis 
and  results  showed  that  approximately  one-third  of  the  variance  in  recruit¬ 
ers’  raw  production  could  be  accounted  for  by  the  variables  studied. 

A  potential  problem  with  both  the  Arima  (1977)  and  Brown  et  al .  (1975) 
studies  has  to  do  with  the  distinction  between  district  membership  as  a 
variable  and  more  specific  environmental  variables  such  as  employment  rate 
and  median  family  income.  District  membership  has  consistently  been  shown 
to  be  a  key  factor  in  recruiter  production,  yet  this  finding  says  nothing 
about  which  aspects  of  different  districts  (or  which  environmental  varia¬ 
bles)  result  in  the  recruiter  production  effects.  That  is,  environmental 
factors  provide  possible  reasons  for  between-district  differences  in  pro¬ 
duction  levels.  Thus,  for  conceptual  reasons,  it  appears  appropriate  to 
keep  separate  consideration  of  specific  environmental  variables  and  dis¬ 
trict  membership.  The  Arima  and  Brown  et  al .  analyses  fail  to  make  this 
distinction,  in  fact  including  both  district  membership  and  environmental 
variables  in  the  same  regression  equations. 

Borman.  Rosse.  and  Toouam  (19821.  To  delineate  the  environmental 
variables  impacting  on  recruiter  production,  these  authors  developed  a  list 
of  potentially  important  factors  from  interviews  with  officers  and  Navy  Re¬ 
cruiting  Command  officials.  They  obtained  seven  factors  including: 

1.  Unemployment  rate  in  the  recruiting  area. 

2.  Level  of  competition  (i.e.,  the  ratio  of  number  of  other  service 
recruiters/number  of  Navy  recruiters). 

3.  Ratio  of  military  to  civilian  pay  in  an  area. 

4.  Ratio  of  military  to  total  population  in  an  area. 

5.  Ratio  of  Department  of  Defense  civilian  employees  to  total  popu¬ 
lation  in  an  area. 

6.  Propensity  of  young  people  to  enlist  in  an  area  (i.e.,  data  from 
the  Youth  Attitude  Tracking  Survey). 

7.  Ratio  of  support  personnel  available  to  process  recruit  data/the 
total  number  of  recruiters. 

Measures  of  these  variables  were  obtained  at  the  district  level  (N  = 
43),  and  mean  production  scores  were  computed  for  these  districts. 

To  examine  relationships  between  the  environmental  variables  and  pro¬ 
duction,  the  authors  correlated  each  variable  with  mean  district  produc¬ 
tion.  Of  the  seven  environmental  factors,  three  correlated  consistently 
with  mean  district  production  across  four  quarters  of  production  data 
(Military/Civilian  Pay  Ratio,  r  =.36;  Unemployment  Rate,  r  =.30;  and  Pro¬ 
pensity  to  Enlist,  r  =.23  against  FY  80  total  production),  indicating  that 
district  level  production  is  affected  by  some  of  the  environmental  factors. 


To  assess  the  impact  of  the  environmental  variables  (in  combination)  on 
individual  recruiter  production  levels,  the  authors  employed  an  ANOVA 
procedure  producing  estimates  of  variance  accounted  for  by  the  environmen¬ 
tal  variables,  by  differences  between  districts,  and  by  within-di strict 
factors.  Although  the  effects  of  the  environmental  variables  on  individual 
recruiter  production  levels  was  significant  (F  =  10.87,  £<.001),  they 
accounted  for  only  five  percent  of  the  total  variance  in  production.  The 
total  variance  accounted  for  by  differences  between  districts  was  7.4 
percent  (after  the  effects  of  the  environmental  measures  were  removed). 
Importantly,  most  of  the  variance  in  recruiter  production  (87.6%)  was  due 
to  within-district  factors.  In  other  words,  the  bulk  of  the  variance  in 
recruiter  production  may  result  from  between-recruiter  differences  in  re¬ 
cruiting  ability  or,  perhaps,  environmental  factors  at  more  fine-grained 
geographical  levels  (e.g.,  zone,  station). 

In  sum,  a  number  of  studies  have  investigated  either  the  personal 
characteristics  that  might  predict  production  or  the  environmental  factors 
that  may  influence  individual  recruiter  productivity.  With  regard  to 
person  characteristics,  non-cognitive  variables  have  validly  predicted 
production  criteria  (cf.  Atwater  et  al.,  1986;  Bennett  &  Haber,  1973; 

Borman  et  al . ,  1980;  Larriva,  1975).  Although  they  have  also  shown  lower 
validities  than  when  ratings  are  used  as  criteria  (cf.  Borman  et  al . ,  1979; 
Brogden  &  Taylor,  1949),  recent  large-scale  validation  research  evaluating 
non-cognitive  predictors  against  production  supports  this  relationship. 

Because  environmental  factors,  as  well  as  the  skills  and  abilities  of 
individual  recruiters,  influence  recruiter  productivity,  research  has 
focused  on  identifying  contaminants  of  production  data.  Brown  et  al . 

(1975)  and  Larriva  (1975)  developed  production  indices  adjusted  to  account 
for  territorial  factors.  Researchers  have  also  worked  toward  defining  the 
environmental  variables  impacting  production  (Arima,  1977;  Borman  et  al., 
1982;  Brown  et  al . ,  1975).  Variables  describing  the  recruiting  setting 
(e.g.,  unemployment  rate,  military/civilian  pay  ratio)  have  produced  signi¬ 
ficant  correlations  with  production  data.  To  date,  research  in  this  arena 
has  focused  on  variables  influencing  production  at  the  district  level 
(approximately  equivalent  to  battalion  level).  Adjusting  production 
indices  for  district,  or  battalion,  differences  does  remove  some  contami¬ 
nating  variance;  however,  there  appears  to  be  considerable  production 
variance  within  districts  (Borman  et  al.,  1982).  Further  research  at  a 
more  fine-grained  level  (e.g.,  the  company  level)  is  needed  before  purer 
production  indices  can  be  developed. 


^Forty-one  percent  of  the  total  variance  between-di stricts  in  production 
was  accounted  for  by  the  seven  environmental  factors. 
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Predicting  Recruiter  Success  in  Training 


Studies  discussed  in  the  preceding  chapters  employed  subjective  perfor¬ 
mance  criteria  (i.e.,  ratings)  or  an  objective  effectiveness  criterion 
(i.e.,  production)  to  define  on-the-job  success  as  a  recruiter.  In  this 
section,  the  studies  discussed  focus  on  criteria  for  recruiter  success  in 
training.  These  criteria  can  be  subjective  or  objective  and  performance  or 
effectiveness  measures.  Yet  training  criteria  are  different  conceptually, 
and  often  methodologically,  from  measures  of  success  on-the-job.  Concep¬ 
tually  speaking,  training  performance  may  include  behaviors  such  as 
studying  or  practicing  that  are  not  highly  related  to  on-the-job  perfor¬ 
mance.  Likewise,  behaviors  important  for  recruiter  performance  on-the-job 
(e.g.,  actions  taken  to  establish  rapport  with  the  community)  may  not  be 
highly  related  to  those  important  for  successful  performance  during 
training.  Thus,  with  respect  to  training  criteria,  the  term  "performance" 
may  imply  different  performance  requirements  than  is  the  case  with  job 
performance. 

Massey  and  Mullins  (1966).  In  this  early  study,  a  dichotomous,  ob¬ 
jective  criterion,  graduation  versus  elimination  from  training,  was  employed 
to  assess  the  validity  of  an  Air  Force  recruiter  selection  battery.  The 
authors  constructed  an  eight-inventory  battery  designed  to  measure  quali¬ 
ties  such  as  empathy,  surgency  (friendliness  and  sociability),  and  perse¬ 
verance,  all  thought  to  be  desirable  in  recruiters.  The  measures  are 
listed  below: 

1.  Airman  Preference  Test- -a  43-item  five-choice  test  designed  to  measure 
empathy. 

2.  CEI  Opinion  Questionnaire--83  items  describing  a  person’s  behavior,  an 
individual’s  opinions,  old  proverbs,  and  expressions  of  interest.  The 
examinee  is  to  show  agreement,  disagreement,  or  no  opinion.  Three 
scores  (Complexity,  Ego  Strength,  and  Introversion)  were  considered 
likely  predictors  of  recruiter  success. 

3.  Community  Information--43  items  to  gain  factual  information  about  areas 
of  former  residence.  This  was  designed  on  the  rationale  that  a  person 
who  knows  more  about  the  community  is  one  who  is  more  interested  in  it 
and  consequently  one  who  should  be  a  more  effective  recruiter  in  that 
location. 

4.  Descriptive  Adjective  Inventorv--182  paired  adjectives  used  to  describe 
a  person’s  general  appearance,  typical  behavior,  and  relations  with 
others.  The  examinee  selected  the  one  adjective  of  the  pair  that  best 
describes  him  or  her.  Respondents  were  scored  on  three  scales  (Sur¬ 
gency,  Cooperativeness,  and  Orderliness). 

5.  FCSRI-A--190  items,  arranged  in  pairs  of  statements,  that  refer  to  a 
person’s  appearance,  typical  behavior,  and  relations  to  others.  Three 
scores  (Surgency,  Cooperativeness,  and  Orderliness,  the  same  three 
variables  as  measured  by  the  Descriptive  Adjective  Inventory)  were 
considered  likely  predictors  of  recruiter  effectiveness. 


6.  Recruiter  Languaae--made  up  of  four  subtests: 
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15  five-choice  items--examinee  must  select  the  initial  letter  of  a 
missing  word.  The  definition  is  provided. 


15  five-choice  items--examinees  must  select  the  correct  arrange¬ 
ment  of  words  appearing  in  progressive  order.  The  words  express 
varying  degrees  of  the  same  general  concept. 


10  five-choice  items--examinee  must  select  the  best  sentence  with 
respect  to  grammar  and  usage. 


15  five-choice  items--a  sentence  is  provided;  an  examinee  must 
select  the  interpretation  that  means  most  nearly  the  same  as  the 
sentence. 


7.  Texas  Social  Insiaht--40-item  test  to  describe  ordinary,  unusual,  or 
embarrassing  situations.  The  examinee  is  to  select  the  alternative 
he/she  sees  as  the  correct  course  of  action. 


8.  Word  Power- -test  of  ability  to  call  to  mind  certain  words.  At  a  given 
signal  the  examinee  records  as  many  words  as  possible  beginning  and 
ending  with  a  given  letter  of  the  alphabet. 


In  a  preliminary  validation  step,  the  battery  was  administered  to  210 
recruiter  students  prior  to  entry  in  the  recruiter  course.  Twenty-four 
scores,  derived  from  the  eight  inventories,  were  correlated  with  the  dicho¬ 
tomous  criterion  measure  of  graduation  or  elimination  from  the  recruiter 
course  (178  graduated  and  32  were  eliminated).  Four  of  the  scores--Re- 
cruiter  Language,  Community  Information,  FCSRI-A  (Surgency),  and  the  Texas 
Social  Insight  Test--yielded  significant  point  biserial  correlations 
(£<.01). 


A  combination  of  the  Recruiter  Language,  Community  Information,  and 
FCSRI-A  (Surgency)  scores  yielded  a  multiple  correlation  of  .23  (jk.01). 
(The  Texas  Social  Insight  Test  did  not  add  significantly  to  the  multiple 
correlation.)  The  three  inventories,  with  scoring  weights  of  2,  1,  and  1 
respectively,  were  employed  as  an  interim  selection  battery,  pending  fur¬ 
ther  validation  of  the  eight-inventory  composite. 


Next,  the  full  eight-inventory  battery  was  administered  to  a  larger 
sample  of  recruiter  students  (N  =  1067)  prior  to  entry  in  the  recruiter 
course.  Background  information  including  age,  education,  marital  status, 
and  number  of  dependents  was  also  collected.  As  the  criterion,  pass/fail 
information  was  subsequently  obtained  for  each  recruiter  student. 


The  validation  sample  (N  =  965,  students  with  complete  information)  was 
randomly  split  into  two  groups--Sample  B  (N  =  485),  the  validation  sample; 
and  Sample  C  (N  =  480),  the  cross-validation  group.  Twenty-four  scores, 
derived  from  the  eight  inventories,  were  generated  for  Sample  B  recruiters. 
These  scores,  along  with  the  background  variables,  were  correlated  with  the 
recruiter  course  pass/fail  criterion.  No  combination  of  predictors  signi¬ 
ficantly  (e<.05)  increased  prediction  over  that  obtained  using  the  three 
scores  from  the  interim  battery.  The  scoring  weights  of  the  interim  bat¬ 
tery  (RSST-63)  were  altered  a  number  of  times  and  usod  to  score  Sample  B 


recruiters.  These  scores  were  validated  against  the  pass/fail  criterion 
and  again,  no  combination  of  regression  weights  surpassed  the  prediction 
capabilities  of  the  interim  battery  with  the  original  weights  (r  =  .34, 
jk.01).  The  regression  weights  of  the  predictor  scores  for  Sample  B  (the 
same  scores  and  weights  used  in  the  interim  battery)  were  also  applied  to 
Sample  C  responses.  The  scores  of  sample  C  recruiters  were  calculated  and 
validated  against  the  pass/fail  criterion  and  yielded  a  point  biserial 
correlation  of  .21  (jx.05). 

The  same  procedures  were  followed  in  validating  the  predictor  scores 
against  a  second  criterion--supervisor  field  ratings.  These  ratings  were 
obtained  from  supervisors  one  year  after  each  recruiter  was  placed  on  duty. 
Results  showed  that  no  combination  of  variables  of  the  entire  set  signifi¬ 
cantly  predicted  the  field  ratings  at  the  .05  level. 

Two  other  variables  outside  the  predictor  battery  were  also  investi¬ 
gated  for  predicting  recruiter  success.  First,  the  recruiter’s  primary  Air 
Force  Specialty  Code  (AFSC),  or  the  service  specialty  in  which  a  recruiter 
had  previously  worked,  was  investigated  because  recruiter  training  person¬ 
nel  believed  that  those  specialties  requiring  more  interpersonal  contact 
would  produce  better  recruiters  than  those  specialties  that  require  little 
contact.  The  primary  AFSC  variable  was  divided  into  four  service  specialty 
variables  reflecting  various  degrees  of  required  interpersonal  contact. 
Additionally,  12  interaction  variables  were  generated  by  multiplying  each 
of  the  four  primary  service  variables  by  aptitude,  surgency,  and  depend¬ 
ability  scores,  thus  creating  12  hybrid  individual  differences/AFSC  varia¬ 
bles.  These  16  variables  were  then  correlated  with  the  field-rating  cri¬ 
terion  measure  for  a  sample  of  859  recruiter  course  graduates.  Only  one  of 
the  16  correlated  as  highly  as  .07  (£<.05).  The  authors  therefore  dis¬ 
carded  the  primary  AFSC  as  a  predictor  of  recruiter  success. 

A  second  variable  regarded  as  a  possible  predictor  of  field  recruiter 
performance  was  recruiter  school  advisor  ratings.  These  ratings,  collected 
while  the  recruiters  were  in  recruiter  school,  consisted  of  a  single  7- 
point  scale  ranging  from  outstanding  to  unsatisfactory.  The  advisor  rat¬ 
ings  were  related  to  field  ratings,  yielding  a  correlation  of  .19  (jk.05). 

The  results  of  this  study  indicate  that  a  battery  consisting  of  the 
Recruiter  Language  Test,  Community  Information  Inventory,  and  the  FCSRI-A 
(Surgency)  can  be  used  to  predict  recruiter  school  performance,  but  that 
neither  this  battery  nor  recruiter  school  advisor  ratings  can  be  used  to 
predict  field  recruiter  performance.  The  authors  attribute  the  failure  to 
predict  field  recruiter  performance  to  the  supervisory  performance  rating 
criterion  measure.  They  suggest  that  this  kind  of  rating  is  inherently 
contaminated  by  several  rater  errors  such  as  leniency.  Another  difficulty 
may  be  that  the  battery  the  authors  developed  is  simply  more  appropriate 
for  predicting  recruiter  school  success  than  field  recruiter  performance. 

At  any  rate,  Massey  and  Mullins  suggested  that  a  more  reliable  and  valid 
criterion  measure  should  be  developed  to  provide  a  fair  assessment  of  the 
utility  of  any  paper-and-pencil  measures  purporting  to  predict  recruiter 
performance. 

Borman  (19821.  All  the  studies  discussed  thus  far  employed  paper-and- 
pencil  measures  to  predict  recruiter  performance  or  effectiveness.  By 
contrast,  Borman  (1982)  developed  assessment  center  exercises  targeted 
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toward  predicting  Army  recruiter  performance  in  training.  The  assessment 
center  exercises  were  designed  to  tap  14  personal  characteristics  (e.g., 
Sociability,  Personal  Impact)  that  experts  on  the  Army  recruiter  job  had 
rated  as  important  for  performing  recruiter  tasks  [Borman,  Toquam,  &  Rosse 
(1976),  provides  a  summary  of  the  initial  job  analysis].  The  six  exercises 
developed  by  Borman  were: 

1.  Structured  interview.  Assessors  ask  a  series  of  questions  targeted  at 
the  subject’s  level  of  achievement  motivation,  potential  for  being  a 
"self-starter",  and  commitment  to  the  Army. 

2.  Cold  calls.  Subject  has  an  opportunity  to  learn  a  little  about  three 
prospects  (e.g.,  Joe  Hill  and  Jack  Brown)  and  must  phone  each  of  them 
for  the  purpose  of  getting  them  to  come  into  the  office.  Assessor  role 
players  have  well-defined  characters  (prospects)  to  portray. 

3.  Interviews.  Two  of  the  three  cold-call  prospects  agree  to  come  in  for 
an  interview.  The  subject’s  job  is  to  follow  up  on  what  was  learned  in 
the  cold-call  conversations  and  to  begin  promoting  Army  enlistment  to 
these  people.  A  third  "walk-in"  prospect  also  appears  for  an  interview 
with  the  subject. 

4.  Interview  with  concerned  parent.  Subject  is  asked  to  prepare  for  and 
conduct  an  interview  with  the  father  of  one  of  the  prospects  he  or  she 
interviewed  previously. 

5.  5-minute  speech  about  the  Army.  Subject  prepares  a  short  talk  about  an 
Army  career  that  he  or  she  delivers  to  the  rest  of  the  group  and  to  the 
assessors. 

6.  In-basket.  Subject  is  given  an  in-basket  filled  with  notes,  phone 
messages,  and  letters  on  which  he  or  she  must  take  some  action. 

Borman  conducted  a  two-day  course  to  train  16  experienced  and  success¬ 
ful  Army  recruiters  as  assessors.  Fifty-seven  soldiers  entering  the  U.S. 
Army  recruiter  school  with  no  previous  recruiting  experience  were  assessed 
in  the  evaluation  program.  Twelve  assessors  were  assigned  in  pairs  to 
subjects  in  the  structured  interview  and  five  role-playing  exercises.  Four 
additional  assessors,  working  in  pairs,  provided  "first  impression"  rat¬ 
ings,  or  evaluations  of  subjects  made  after  only  brief  exposure  to  them 
during  a  15-minute  introductory  session  at  the  beginning  of  the  assessment 
day.  The  author  made  physical  attractiveness  and  likeability  ratings  of 
each  subject  on  the  basis  of  interaction  with  them  in  the  introductory 
meeting  and  during  exercise  instruction  sessions  with  them  before  each 
exercise. 

Assignments  of  assessors  to  subjects  and  exercises  were  made  such  that 
(1)  assessor  pairs  never  worked  together  more  than  one  day  during  the  ten- 
day  program  to  avoid  possible  rater  effects  on  interrater  agreement  re¬ 
sults;  (2)  assessors  were  assigned  to  the  main  group  of  12  for  seven  or 
eight  of  the  ten  days  and  to  the  group  of  four  for  the  other  days;  and  (3) 
each  pair  viewed  each  subject  in  one  and  only  one  of  the  six  exercises 
during  the  day.  This  latter  feature  provided  the  opportunity  to  evaluate 
the  relative  contribution  of  each  exercise  to  validity. 


For  each  exercise,  assessors  rated  each  trainee  on  the  personal  charac¬ 
teristics  relevant  for  that  exercise  and  provided  an  overall  performance 
rating  for  each  exercise.  At  the  end  of  each  day,  the  assessors  discussed 
subjects’  performance  and  reached  a  consensus  on  their  ratings.  Thus,  the 
assessment  center  design  provided  five  types  of  predictor  measures: 

1.  First  impression,  physical  attractiveness,  and  likeability  ratings; 

2.  Ratings  on  a  structured  interview; 

3.  Ratings  of  subjects’  performance  on  individual  exercises; 

4.  Consensus  ratings  of  subjects’  performance;  and, 

5.  A  statistical  composite  (unit  weighted)  of  the  assessment  ratings. 

Additionally,  Borman  administered  the  SAB  (Borman  et  al . ,  1980)  and  a 
personality  inventory  (Brown  et  al.,  1975)  as  predictor  measures. 

Three  measures  obtained  during  two  phases  of  training  were  employed  as 
criteria.  A  Phase  1  composite  of  scores  on  three  objective  tests  comprised 
one  criterion.  The  tests  measured  mastery  of  prospecting  and  selling 
techniques  taught  in  Phase  1.  An  intraclass  correlation  of  .94,  indexing 
the  composite’s  reliability,  suggested  that  the  Phase  1  criterion  measures 
were  homogeneous. 

In  Phase  2,  students  practiced  the  telephoning  and  interviewing  techni¬ 
ques  they  had  learned  in  Phase  1.  In  small  groups  and  with  the  help  of  a 
staff  instructor,  they  practiced  their  skills  in  hypothetical  recruiting 
situations.  Between  the  fifth  and  the  eighth  day  after  students  had  en¬ 
tered  Phase  2,  instructors  rated  these  students  on  three  performance  dimen¬ 
sions.  In  general,  individuals  who  were  performing  best  in  Phase  2  were 
released  earliest  (r  =  -.52  between  a  composite  of  the  Phase  2  instructor 
ratings  and  number  of  days  to  complete  Phase  2).  The  Phase  1  composite 
correlated  .72  and  -.33  respectively  with  instructor  evaluations  in  Phase  2 
and  time  to  complete  Phase  2.  It  should  be  noted  that  instructors  in  the 
training  course  had  no  contact  with  the  assessment  program  or  the  as¬ 
sessors.  Thus,  ratings  they  made  of  trainee  performance  during  training 
were  completely  independent  of  assessment  ratings. 

Interrater  agreement  between  assessors  on  each  personal  characteristic 
for  each  of  the  six  exercises  ranged  from  .44  to  .92  with  a  median  of  .76. 
Ratings  of  overall  performance  were  more  reliable  (median  N  =  .84)  than 
ratings  on  individual  personal  characteristics.  Validity  coefficients 
between  the  predictors  and  three  criteria  are  presented  in  Table  6. 

As  sl,own,  first  impression,  physical  attractiveness,  and  likeability 
ratings  showed  nonsignificant  validities,  as  did  the  structured  interview, 
the  SAB,  and  the  personality  test.  On  the  other  hand,  ratings  on  all  five 
role-playing  exercises  yielded  correlations  of  .32  (]X.05)  or  greater  with 
the  Phase  1  composite.  Correlations  were  not  as  high  for  the  Phase  2 
criteria,  performance  ratings  and  time-to-completion. 


TABLE  6 

Correlations  Between  Assessment  Measures  and  Training  Criteria 

(Borman,  1982) 


Variable 

Phase  1 

Phase  2 

Composite 

performance 

Composite 

performance 

Time  to 
complete 

Special  Assignment  Battery  (SAB) 

test  keys  (Navy) 

Human  relations 

-06 

15 

-27 

Selling 

02 

17 

-12 

Organizing 

-01 

07 

-15 

Overall  performance 

-09 

15 

-09 

Army  personality  test 

Experimental  key 

04 

05 

-21 

Assessment  measures 

First  impression 

23 

-18 

-02 

Physical  attractiveness 

-06 

-20 

06 

Likeabil ity 

06 

04 

14 

Structured  interview 

07 

26 

00 

Assessment  ratings 

Cold  calls 

33* 

-03 

-26 

Interviews 

34* 

15 

-28 

Interview  with  parent 

41** 

24 

-29* 

In-basket 

33* 

36* 

-11 

Speech 

32* 

27 

-42** 

Consensus  rating 

38** 

27 

-24 

Unit-weighted  composite 

Ratings  on  relevant 

personal  characteristics  48**  35*  -33* 

(all  exercises) 

Corrected  for  attenuation  53**  50**  -49** 


Note:  For  Phase  1,  N  =  50;  For  Phase  2,  N  =  46. 
Decimal  points  are  omitted. 

*  £<.05  (two-tailed).  **  £<.01  (two-tailed). 
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The  unit-weighted  composite  of  the  assessment  (i.e.,  the  sum  of  the 
scores  on  each  personal  characteristic  pooled  across  the  six  exercises) 
correlated  significantly  with  each  of  the  three  criteria  (.48  with  the 
Phase  1  composite;  .35  with  the  Phase  2  composite;  and  -.33  with  time-to- 
complete  Phase  2).  Borman  noted  that  these  validities  were  conservative 
because  attrition  occurred  during  the  study.  Four  trainees  with  relatively 
low  assessment  scores  were  eliminated  from  training  because  of  inadequate 
Phase  1  performance.  Seven  persons  who  received  either  very  high  or  very 
low  assessment  ratings  dropped  out  before  training.  Thus,  the  range  of 
assessment  ratings  was  restricted.  Corrected  for  restriction  of  range, 
validity  coefficients  for  the  unit-weighted  composite  were  .53,  .50, 
and  -.49,  respectively  against  the  three  criterion  measures. 

Borman.  Eaton.  Bryan,  and  Rosse  (19831.  This  study  further  explored 
the  validity  of  the  assessment  exercises  developed  by  Borman  (1982)  for 
predicting  success  in  recruiter  training.  A  subset  of  the  exercises  em¬ 
ployed  previously  by  Borman  was  administered  to  450  sergeants  entering  the 
U.S.  Army’s  recruiter  school.  These  exercises  included: 

1.  "Cold  call"  with  Joe  Hill,  a  prospective  recruit  unwilling  to  consider 
Army  enlistment; 

2.  "Cold  call"  with  Jack  Brown,  a  prospect  willing  to  consider  Army 
recruitment; 

3.  5-minute  speech  about  the  Army;  and, 

4.  In-basket  exercise. 

Thirteen  experienced  recruiters  received  assessor  training  and  served 
as  assessors.  For  each  exercise,  assessors  evaluated  assessees  on  the 
personal  characteristics  relevant  for  that  exercise  and  on  an  overall 
performance  dimension. 

Similar  to  the  1982  study,  three  training  performance  criteria  served 
as  elements  of  the  criterion  composite.  One  element  was  a  100-item  objec¬ 
tive  test  measuring  mastery  of  prospecting  and  selling  techniques  taught  in 
the  first  phase  of  recruiter  school.  The  second  and  third  elements  were 
instructor  ratings  of  telephone  and  interview  selling  techniques  taught  in 
a  second  phase  of  recruiter  training.  The  telephone  and  interview  perfor¬ 
mance  ratings,  made  by  different  instructors,  correlated  .28,  and  each  set 
of  ratings  correlated  .24  with  the  test  scores.  A  unit-weighted  composite 
of  these  three  criteria  was  used  as  the  primary  criterion  in  this  research. 

Correlations  between  the  assessment  ratings  and  training  performance 
scores  were  computed  to  investigate  the  validities  of  the  various  exercises 
as  predictors  of  training  performance  criteria.  The  validities  for  the 
four  individual  exercises  ranged  from  .10  to  .26,  with  a  median  of  .19. 

The  correlation  between  the  unit  weighted  composite  of  the  exercise  ratings 
and  the  composite  criterion  was  .32.  Although  these  validities  were  not 
impressively  high,  they  were  significantly  different  from  zero  (jk.OI). 


Borman  et  al .  (1983)  suggest  that  the  lower  validities  obtained  in  this 
study  compared  to  previous  work  (Borman,  1982)  may  have  been  the  result  of 


changes  in  the  performance  criterion  measures  taken  at  the  recruiter 
school.  Another  possibility  is  that  the  assessors  may  have  been  distracted 
from  predicting  performance  by  their  additional  responsibilities  of  provid¬ 
ing  performance  feedback  after  each  exercise. 

In  sum,  the  validity  of  both  paper-and-pencil  measures  and  assessment 
scores  for  predicting  recruiter  success  in  training  has  been  investigated. 
Massey  and  Mullins  (1966)  found  that  paper-and-pencil  measures  of  person¬ 
ality  characteristics  and  verbal  ability  successfully  predict  completion  of 
recruiter  training  (pass/fail).  Paper-and-pencil  measures  of  non-cognitive 
variables  did  not,  however,  correlate  significantly  with  objective  training 
test  scores,  instructor  ratings,  or  time  to  complete  training  criteria 
employed  by  Borman  (1982).  Borman  also  designed  assessment  center  exer¬ 
cises  to  tap  non-cognitive  and  cognitive  personal  characteristics  and 
employed  assessment  scores  as  predictors.  Because  both  non-cognitive  and 
cognitive  characteristics  are  tapped  by  such  exercises,  no  conclusion 
regarding  the  relative  merit  of  either  type  of  variable  as  a  predictor  can 
be  drawn.  Of  particular  importance  though,  this  research  provides  evidence 
for  the  utility  of  the  assessment  approach  for  predicting  both  objective 
and  subjective  training  criteria. 


Measuring  and  Predicting  Performance  In  Military 
Recruiter  and  Civilian  Sales  Jobs: 

An  Integration 


In  previous  sections  we  described  research  conducted  to  identify  the 
person  characteristics  and  environmental  factors  associated  with  military 
recruiter  success.  At  a  broader  level,  however,  the  military  recruiter  job 
is  one  among  many  sales  jobs,  and  within  the  civilian  sales  arena  numerous 
studies  have  been  conducted  to  delineate  variables  related  to  sales  effec¬ 
tiveness.  We,  therefore,  prepared  a  review  of  civilian  sales  selection 
research  (Kanfer  &  Borman,  1986)  and  analyzed  the  similarities  and 
differences  among  factors  underlying  military  recruiter  and  civilian  sales 
selection  research. 

Three  major  themes  emerged  from  our  comparison  of  these  two  lines  of 
research  (i.e.,  military  recruiter  and  civilian  sales  selection).  First, 
although  recruiting  is  essentially  selling,  differences  in  the  type  of 
"product"  sold  and  clientele  served  result  in  some  differences  between  the 
military  recruiter  job  and  sales  jobs  in  the  civilian  sector.  For  instance, 
recruiters  are  selling  a  lifestyle  or  a  career  rather  than  material  goods 
or  services.  Such  differences  between  military  and  civilian  sales  jobs  lead 
to  somewhat  different  behavioral  requirements  in  the  two  arenas  and,  in 
turn,  can  moderate  the  validity  of  specific  predictors  against  performance 
criteria.  The  usefulness  of  civilian  sales  selection  research  for  drawing 
detailed  conclusions  about  the  variables  likely  to  predict  military 
recruiter  performance  is,  thus,  somewhat  limited. 

Second,  the  major  points  regarding  the  quality  of  various  criteria 
discussed  throughout  previous  chapters  of  this  review  apply  in  both 
civilian  and  military  sales  arenas.  That  is,  environmental  factors  may 
restrict  or  enhance  the  range  of  possible  outcomes  of  performance,  and  thus 
contaminate  objective  effectiveness  measures,  and  objective  indices  may  not 
tap  all  aspects  of  performance.  Subjective  measures  (such  as  supervisor 
ratings)  can  provide  multidimensional  evaluations,  are  probably  not 
restricted  by  environmental  factors,  and  thus  offer  a  distinct  advantage  in 
selection  research.  The  quality  of  subjective  measures  is,  however, 
influenced  by  the  source  of  the  rating  and  various  rating  errors  and 
biases.  Cumulating  research  across  civilian  and  military  jobs  is,  thus, 
useful  for  identifying  potential  pitfalls  and  problems  associated  with 
various  criteria,  understanding  the  interrelationships  among  criteria,  and 
developing  a  strategy  for  obtaining  criterion  data  of  high  quality. 

Finally,  the  major  finding  of  our  comparison  is  that  some  of  the 
personal  attributes  that  lead  to  success  in  recruiting  are,  in  general, 
similar  to  those  requisite  to  effective  civilian  sales  performance.  In 
particular,  personality  measures  tapping  dominance  or  forcefulness  appear 
to  be  associated  with  both  recruiter  and  sales  success,  and  in  contrast, 
aptitude  measures  have  little  support  as  predictors  of  success  in  either 
arena.  Vocational  interest  variables  frequently  found  to  be  good 
predictors  of  military  recruiter  success  have  not  been  researched  suffi¬ 
ciently  against  civilian  sales  performance.  Similarly,  little  research  has 
investigated  the  validity  of  skill  level  variables  (e.g.,  objective 
assessment  of  interpersonal  skills)  for  predicting  military  recruiter  job 


performance.  Because  such  variables  have  yielded  reasonably  good 
validities  in  sales  selection  research,  they  may  also  be  useful  predictors 
of  recruiter  job  performance. 

The  following  three  sections  summarize  recruiter  selection  research 
described  previously  in  this  review,  integrating  relevant  research  from  the 
civilian  sector.  In  particular,  the  first  section  discusses  simil iarities 
and  differences  between  civilian  sales  and  military  recruiter  jobs,  and  the 
second  section  focuses  on  criterion-related  research  to  explicate  future 
research  issues.  The  third  section  summarizes  validation  research  for 
specific  predictor  variables  in  both  military  and  civilian  sales  settings. 

The  Performance  Domain 

How  similar  are  the  performance  requirements  of  the  military  recruiter 
job  to  those  of  civilian  sales  jobs?  In  the  broadest  sense,  all  sales  jobs 
involve  behaviors  on  the  part  of  the  salesperson  to  sell  and  promote  a 
product  to  a  client.  Differences  in  the  behaviors  requisite  to  effective 
performance  in  sales  jobs,  however,  may  emerge  from  differences  in  the 
types  of  products  sold  or  the  types  of  clientele  served. 

Sales  jobs  are  extremely  diverse  in  terms  of  the  types  of  products 
sold  (e.g.,  life  insurance,  computers,  services)  and  the  clientele  served 
(e.g.,  individual  customers,  businesses).  How  does  this  diversity  impact 
on  predictor-criterion  relationships?  In  a  recent  meta-analysis  of 
civilian  sales  research,  Churchill  et  al .  (1985)  categorized  products  as 
consumer  goods,  industrial  goods,  or  services,  and  found  product  type  to 
moderate  the  relationship  between  predictors  and  criteria.  More  specifi¬ 
cally,  product  type  had  the  greatest  moderating  effect  on  the  relationship 
between  non-cogniti ve  variables  (e.g.,  motivation,  personality)  and  sales 
success  measures.  The  authors  also  investigated  moderating  effects  of 
customer  type  (independent  of  product  type)  by  classifying  customers  as 
either  individual  consumers  or  institutions.  The  results  suggested  that 
customer  type  moderates  relationships  between  non-cognitive  predictors  and 
measures  of  sales  success. 

Two  aspects  of  this  work  warrant  further  attention.  First,  the 
product  and  customer  types  categories  employed  by  Churchill  et  al .  (1985) 
are  quite  broad.  Such  breadth  could  attenuate  the  moderating  effects  of 
these  variables  by  increasing  the  variance  within  moderator  type.  Second, 
the  product  type  and  customer  type  variables  employed  in  this  work  are 
tapping  redundant  information.  For  example,  most  industrial  goods  are 
probably  sold  to  institutions.  Perhaps  categorizing  products  more  speci¬ 
fically  (e.g.,  vehicles,  appliances,  insurance/benefits,  etc.)  would 
further  distinguish  moderating  effects  of  these  variables.  Of  particular 
importance,  though,  this  research  suggests  that  product  and  client  types 
moderate  predictor-criterion  relationships  even  when  these  factors  are 
categorized  broadly. 

With  respect  to  product  and  client  type  the  military  recruiter  job  is 
quite  different  from  most  sales  jobs  in  the  civilian  sector.  Indeed,  mili¬ 
tary  recruiters  are  selling  a  concept,  a  career,  a  lifestyle,  or  an  educa¬ 
tion  rather  than  a  "product."  On  the  "client"  side,  military  recruiters 
are  faced  with  selling/projecting  an  image  of  the  armed  services  to  the 
community,  as  well  as  selling  military  careers  to  young  adults  individually. 


Because  products  and  clients  define  many  of  the  knowledges  required  by 
the  sales  job,  they  impact  on  behaviors  in  which  job  knowledge  is  exhi¬ 
bited.  Additionally,  specific  products  or  clients  may  necessitate  a  unique 
set  of  job  behaviors.  To  explicate  these  ideas,  task  and  performance 
categories  developed  through  job  analyses  of  Navy/Marine  Corps  (Borman  et 
al . ,  1976)  and  Army  (Borman,  Toquam,  &  Rosse,  1976b)  recruiter  jobs  are 
provided  in  Table  7.  The  four  Army  recruiter  task  dimensions  resulted  from 
an  empirical  analysis  of  task  responses  (Borman  et  al.,  1976b)  and  actually 
subsume  26  more  specific  task  categories.  The  eight  Navy/Marine  Corps 
recruiter  performance  dimensions  were  developed  by  the  critical  incident  or 
behavior  analysis  methodology  and  were  described  in  an  earlier  section  of 
this  review.  Also  included  in  Table  7,  are  recent  results  of  Army 
recruiter  performance  rating  scale  development  research  (Borman,  Russell,  & 
Skilling,  1986).  Consistently  across  these  studies  we  have  found  that 
military  recruiter  performance  involves  a  number  of  selling  behaviors  such 
as  prospecting,  developing  rapport  with  clients,  and  making  sales  presenta¬ 
tions,  and  a  number  of  "other"  behaviors.  The  "other"  behaviors  focus  on 
community  relations,  organizational  and  co-worker  commitment,  planning  and 
organizing,  and  exhibiting  knowledge  of  the  programs  available. 

For  comparison  purposes,  two  job  analyses  of  civilian  sales  positions 
(Behrman  &  Perreault,  1982;  Kirchner  &  Dunnette,  1957)  are  also  included  in 
Table  7.  The  individual  customer-oriented  sales  activities  appear  similar 
across  these  jobs.  One  difference  between  performance  requirements  of  the 
two  (i.e.,  recruiter  and  civilian  sales)  is  the  emphasis  on  community- 
oriented  sales  behaviors  in  the  military  recruiter  job.  More  specifically, 
civilian  sales  performance,  does  not  appear  to  emphasize  publicizing  the 
organization  and  maintaining  rapport  with  persons  in  the  community. 

Perhaps  the  nature  of  recruiter  "products"  (i.e.,  lifestyle,  career)  and 
clients  (i.e.,  young  adults  in  the  community)  requires  community-oriented 
selling  behaviors  as  well  as  individual  customer-oriented  selling 
activities. 

In  summary,  civilian  sales  jobs  are  diverse  in  terms  of  "products  or 
clients,"  and  unlike  most  civilian  sales  jobs,  the  military  recruiter  job 
does  not  involve  selling  a  product  or  a  service.  Differences  in  product 
and  client  type  probably  lead  to  unique  performance  requirements  across 
civilian  sales  jobs  (as  well  as  between  military  recruiter  and  civilian 
sales  jobs)  and,  in  turn,  impact  on  predictor-criterion  relationships. 

Measuring  Performance  and  Effectiveness 

In  addition  to  variations  in  performance  requirements  across 
sales  positions,  methods  of  measuring  performance  or  effectiveness  may 
contribute  variation  in  predictor-criterion  relationships.  Performance  is 
typically  measured  using  subjective  ratings  (e.g.,  peer,  self,  and/or 
supervisory  ratings).  Effectiveness  measures,  on  the  other  hand,  are 
objective  indices  of  a  "bottom-line"  outcome  of  performance  (e.g.,  sales 
volume,  recruiter  production).  In  previous  sections  of  this  review,  we 
discussed  issues  regarding  objective  and  subjective  criteria  and  military 
recruiter  research  that  addresses  these  issues.  The  following  paragraphs 


summarize  major  findings  regarding  performance  measurement  in  the 
recruiting  setting,  incorporating  information  from  research  in  the  civilian 
sector,  and  focusing  on  implications  regarding  future  research. 

Objective  Measures 

Although  "bottom-line"  effectiveness  measures  such  as  sales  volume  and 
recruiter  production  are  objective  criteria,  their  usefulness  in  selection 
research  is  limited  by  two  problems.  First,  as  a  single  index,  an  effec¬ 
tiveness  measure  may  not  tap  all  aspects  of  the  performance  domain.  Se¬ 
cond,  effectiveness  measures  may  be  affected  by  environmental  factors  not 
within  the  individual’s  control.  As  a  consequence,  such  measures  may  not 
accurately  reflect  variation  in  sales  effectiveness  due  to  individual 
differences  in  behavior  and/or  performance. 

What  does  an  effectiveness  criterion  measure?  Many  objective  measures 
have  been  employed  in  civilian  sales  research  (e.g.,  sales  volume  ranking, 
dollar  volume  of  yearly  sales,  earnings,  commissions,  average  number  of 
sales),  and  observed  correlations  among  different  measures  are  sometimes 
high  and  sometimes  low  (cf.  Kanfer  &  Borman,  1986).  Low  correlations  could 
indicate  either  unreliability  (if  the  measures  are  indexing  the  same  con¬ 
struct)  or,  if  the  measures  are  expected  to  tap  different  aspects  of  sales 
success,  actual  differences  in  the  constructs  measured.  Are  different 
objective  measures  indexing  different  aspects  of  sales  success?  What 
relationships  exist  between  objective  indices  and  subjective  measures?  A 
small  number  of  studies  have  examined  such  relationships.  Rush  (1953) 
obtained  correlations  between  several  objective  and  subjective  sales  crite¬ 
ria  and  factor  analyzed  the  correlation  matrix.  Borman  et  al .  (1979) 
factor  analyzed  correlations  between  Navy  recruiter  rating  dimensions, 
resulting  in  an  interpretable  three-factor  solution.  Factor  scores  on  each 
rating  factor  (Selling,  Human  relations,  and  Organizing  Skills)  were  then 
correlated  with  a  summary,  objective  production  index,  and  the  latter  index 
related  most  highly  with  the  Selling  factor,  as  hypothesized.  However, 
further  research  focusing  on  relationships  between  different  objective 
measures  and  between  objective  and  subjective  criteria  is  needed  to  clarify 
factors  that  underlie  sales  performance. 

What  approaches  might  enhance  the  quality  of  objective  measures?  A 
number  of  researchers  in  both  the  recruiter  (e.g.,  Brown  et  al .  1975; 
Larriva,  1975)  and  civilian  sales  (e.g.,  Miner,  1962)  arenas  have  developed 
adjusted  indices  to  reduce  the  contaminating,  non-performance  related  var¬ 
iance  in  effectiveness  criteria.  It  is,  however,  difficult  to  assess  the 
validity  of  an  adjusted  measure  because  it  is  typically  not  known  whether 
the  adjustment  is  reducing  error  or  "true  score"  variance  (or  both).  The 
rationale  behind  the  type  of  adjustment  is,  therefore,  very  important  to 
the  construct  and  content  validity  of  the  index. 

One  common  adjustment  made  on  objective  effectiveness  indices  is  to 
adjust  for  mean  differences  in  production  between  recruiting  terri¬ 
tories.  Referring  to  the  point  made  above,  unfortunately,  to  the  extent 
there  are  meaningful,  "true"  differences  in  individuals’  performance  be¬ 
tween  territories,  such  an  adjustment  removes  actual  variance  in  effective¬ 
ness.  Research  suggests  that  adjusting  for  differences  between  territories 
does  remove  some  contaminating  variance  due  to  the  environment;  however,  it 
is  also  possible  that  environmental  factors  contribute  contaminating  var- 
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iance  within  territories  (Borman  et  al . ,  1982).  Another  way  to  adjust 
objective  measures  is  to  identify  environmental  factors  beyond  the  sales¬ 
person’s  or  recruiter’s  control  and  then  partial  out  of  the  production  index 
variance  due  to  these  variables. 


Regarding  environmental  factors,  recall  that  objective  effectiveness 
indices  are  measures  of  an  organizational  outcome  of  performance.  Envir¬ 
onmental  factors  are  characteristics  of  the  sales  territory  (or  recruiting 
area)  that  restrict  or  enhance  the  range  of  possible  outcomes  (effec¬ 
tiveness).  Research  has  been  conducted  to  delineate  environmental  varia¬ 
bles  (e.g.,  Cravens  &  Woodruff,  1973;  Borman  et  al . ,  1982).  Examples  from 
the  case  of  military  recruiting  are:  (1)  unemployment  rate,  (2)  military/ 
civilian  job  pay  ratio,  and  (3)  propensity  to  enlist.  To  the  extent  dif¬ 
ferent  territories  are  very  different  with  regard  to  these  environmental 
variables,  production  in  individual  territories  might  be  either  enhanced  or 
inhibited.  Unfortunately,  researchers  (e.g.,  Arima,  1977;  Brown  et  al . , 
1975)  have  sometimes  failed  to  distinguish  between  the  territory  itself  as 
a  variable  and  the  more  specific  variables  describing  the  recruiting/sel¬ 
ling  setting,  confusing  the  issue.  Borman  et  al .  (1982)  point  out  that 
adjusting  for  differences  in  territory,  by  definition,  accounts  for  all 
between-territory  environmental  differences.  In  contrast,  the  environment¬ 
al  variables  considered  in  a  study  can  only  approach  accounting  for  all 
between-territory  variance.  In  this  sense,  simply  adjusting  for  mean 
territory  differences  is  better  than  the  partialling  procedure  used  with 
environmental  variables  included  in  the  study.  The  only  problem  with  this 
method  of  adjustment  is  that,  as  mentioned,  it  may  remove  true  performance 
as  wel 1  as  error  (environmental)  variance  from  production  scores.  More 
research  is  needed  to  examine  these  issues,  but  the  researcher  using  pro¬ 
duction  data  should  be  aware  of  these  possible  pitfalls  and  problems  with 
different  adjustment  strategies. 

Subjective  Measures 

Subjective  ratings  of  sales  performance  have  also  been  employed  as 
criteria  in  both  recruiter  and  civilian  sales  selection  research.  Because 
such  ratings  can  provide  multidimensional  evaluations  of  performance  and 
are  not  restricted  by  environmental  factors,  they  offer  a  distinct  advan¬ 
tage  in  selection  research.  The  quality  of  subjective  ratings  is,  however, 
influenced  by  the  source  of  the  rating  and  various  ratinq  errors  and 
biases. 

How  does  the  source  of  the  rating  affect  rating  quality?  Summarizing 
the  advantages  and  disadvantages  of  various  rating  sources,  Kanfer  and 
Borman  (1986)  state: 

"Subjective  ratings  may  be  obtained  from  managerial  personnel,  peers, 
independent  observers,  and/or  self-report.  Each  rating  source  has  its 
advantages  and  disadvantages.  Supervisory  ratings  typically  come  from 
persons  with  good  experience  in  rating  performance,  but  supervisors 
may  be  less  familiar  than  other  sources  with  ratees’  day-to-day  activ¬ 
ities.  Peers  are  usually  very  familiar  with  incumbents’  actual  behav¬ 
ior  related  to  performance,  but  they  often  have  less  experience  (than 
supervisors)  in  completing  ratings  and  are  more  likely  to  provide 
lenient  (higher  than  deserved)  evaluations.  Observers  may  be  less 
prone  to  common  rating  errors  (e.g.,  halo  and  leniency)  and  be  more 


objective,  but  they  lack  the  longer  term  knowledge  of  ratees’  perfor¬ 
mance  over  time.  Finally,  self-ratings  obviously  come  from  a  source 
that  should  know  well  the  true  criterion  performance  levels,  but  these 
ratings  tend  to  be  inflated  (i.e.,  lenient)"  (p.  11). 

How  might  the  quality  of  subjective  ratings  be  enhanced?  Ratings  are 
typically  made  along  a  number  of  dimensions.  When  such  dimensions  (e.g., 
leadership)  are  not  defined  by  the  rating  scale,  individual  raters  may 
adopt  different  definitions  and  standards  for  performance.  Thus,  theoret¬ 
ically,  rating  scales  that  specifically  define  the  behavioral  performance 
requirements  or  standards  within  a  dimension  and  describe  the  behaviors  asso¬ 
ciated  with  various  levels  of  performance  should  lead  to  less  subjectivity 
in  ratings.  Format  comparison  research  has  not  demonstrated  a  clear  super¬ 
iority  of  any  one  specific  type  of  approach  to  designing  formats  (e.g., 

Landy  &  Farr,  1980).  However,  considerable  research  in  applied  settings 
where  "for-research-only"  peer  or  supervisor  ratings  have  been  gathered 
using  behaviorally  oriented  scales  have  resulted  in  reasonably  reliable 
ratings,  demonstrating  some  degree  of  construct  validity,  as  well  (Borman 
et  al . ,  1979,  1980;  Bownas,  Rosse,  &  Dunnette,  1977;  Peterson  &  Houston, 

1980;  Pulakos  &  Borman,  1986).  Interrater  rel iabil ities  typically 
reach  .60-. 70,  factor  analysis  results  are  often  interpretable,  such  as 
with  the  Selling,  Human  Relations,  and  Organizing  Skills  factors  identified 
in  the  Navy  recruiter  research,  and  correlations  between  rating  factors  and 
other  criteria  as  well  as  with  predictors  have  been  conceptually  appro¬ 
priate  (e.g.,  Selling  Skills  correlating  more  highly  with  production  than 
the  other  two  rating  factors  and  the  personality  predictor  scale,  Order, 
relating  higher  with  Organizing  Skills  than  with  the  other  rating  factors 
in  the  same  study).  To  clarify,  we  are  not  asserting  that  ratings  have  no 
or  even  very  few  problems.  Difficulties  with  rating  errors  should  be 
acknowledged  (e.g.,  leniency,  halo,  and  the  like).  However,  steps  can  and 
have  been  taken  to  improve  the  reliability  and  validity  of  performance 
ratings. 

These  positive  results  with  ratings  seem  to  be  a  function  of  the 
combination  of  behavior-based  scales  and  training  (e.g.,  Pulakos  &  Borman, 
1985)  to  orient  raters  to  the  format  and  to  reduce  certain  rating  errors. 
While  no  experimental  results  are  available  to  conclude  that  this  format 
and  training  "treatment"  is  necessarily  most  effective,  it  is  reassuring 
that  criterion  measurement  using  this  combination  has  proven  to  be  reason¬ 
ably  successful  in  actual  practice. 

We  have  discussed  several  factors  contributing  to  variance  in  measures 
of  performance.  First,  product  and  client  types  probably  contribute  some 
true  differences  in  the  behaviors  required  for  effective  performance  across 
sales  positions.  Second,  error  in  subjective  ratings  can  contribute  to 
error  in  these  performance  criteria.  However,  steps  can  be  taken  to  reduce 
some  of  this  error.  Third,  environmental  factors  may  restrict  or  enhance 
the  range  of  possible  outcomes  of  performance  and,  thus,  contaminate  objec¬ 
tive  effectiveness  measures,  and,  finally,  objective  indices  may  not  tap 
all  aspects  of  performance.  All  of  these  factors  could  attenuate  observed 
predictor-criterion  relationships  across  sales  and  recruiter  positions. 
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Personal  Attributes  Important  for  Effective  Performance 


Individual  differences  variables  used  to  predict  the  performance  of 
sales  personnel  and  military  recruiters  can  be  classified  into  five  cate¬ 
gories:  biographical,  personality,  interest,  skill-related,  and  aptitude 
variables.  The  paragraphs  below  briefly  summarize  military  recruiter  and 
civilian  sales  selection  research  with  respect  to  these  predictors.  More 
information  on  military  recruiter  selection  studies  is  provided  in  previous 
chapters  of  this  review,  and  civilian  sales  selection  research  is  discussed 
in  greater  detail  by  Kanfer  and  Borman  (1986). 

Biographical  Variables 

Biographical  variables  represent  a  diversity  of  items  such  as  age, 
prior  selling  experience,  marital  status,  work  habits,  and  education  level. 
Unlike  measures  of  personality,  interest,  and  aptitude  in  which  multiple 
items  tap  a  construct,  a  single  biographical  item  is  often  examined  as  a 
predictor. 

Recruiter  research.  Biographical  variables  have  met  with  some  success 
as  predictors  of  military  recruiter  performance.  Individual  variables  such 
as  educational  background,  prior  service  classification,  marital  status 
(e.g.,  Brogden  &  Taylor,  1949;  Krug,  1972),  work  habits,  and  styles  of 
handling  finances  and  debts  (Brown  et  al ..  1975)  have  sometimes  produced 
significant  correlations  with  performance;  however,  further  research  is 
needed  to  replicate  these  results.  The  Special  Assignment  Battery  (SAB) 
includes  136  biographical  items.  Of  these  only  seven  have  consistently 
yielded  good  relationships  with  recruiter  performance  (Borman  et  al . ,  1979; 
1980;  1983).  These  items  pertain  to  planning  ahead,  involvement  in  and 
attitude  toward  work,  past  recognition  from  teachers,  number  of  towns  lived 
in,  and  attentiveness  to  others.  Interestingly,  they  tend  to  be  related  to 
performance  in  organizing  and  less  associated  with  other  aspects  of 
recruiter  performance. 

Civilian  sales  research.  Aggregating  across  types  of  sales  jobs  and 
types  of  biographical  variables,  Churchill  et  al .  (1985)  obtained  a  mean 
corrected  correlation  of  .29  between  biographical  factors  and  performance 
criteria  in  a  recent  meta-analysis  of  civilian  sales  selection  research. 

The  authors  also  found  that  the  type  of  product  moderated  biographical 
predictor-criterion  relationships.  More  specifically,  corrected  mean  cor¬ 
relations  were  .43,  .29,  and  .21  for  jobs  involving  service  sales,  consumer 
goods,  and  industrial  goods  respectively.  Kanfer  and  Borman  (1986),  sug¬ 
gested  that  these  relationships  might  be  attenuated  by  aggregation  across 
biographical  items  that  may  tap  different  underlying  constructs  and  exam¬ 
ined  results  for  individual  biographical  predictors.  Age,  family  history 
and  status,  education  level,  previous  selling  experience,  and  other  bio¬ 
graphical  variables,  however,  did  not  appear  to  consistently  predict  sales 
performance. 

Summary.  Recruiter  and  civilian  sales  research  suggests  that  bio¬ 
graphical  predictors  may  be  useful  predictors  of  sales  performance.  Re¬ 
search  to  date,  however,  has  not  delineated  which  biographical  variables 
are  predictive  of  sales  success  in  certain  types  of  sales  jobs.  Findings 
for  individual  variables  such  as  education  level  and  family  history  are 
somewhat  inconsistent  or  unreplicated  within  as  well  as  across  these  two 


lines  of  research. 


Three  conceptual  issues  preclude  broad  generalizations  about  the  util¬ 
ity  of  biographical  variables,  in  general,  as  predictors  of  sales  success. 
First,  research  suggests  that  biographical  predictor-criterion  relation¬ 
ships  are  moderated  by  the  type  of  product.  Aggregating  across  types  of 
sales  jobs  is  thus  likely  to  obscure  actual  predictor-criterion  relation¬ 
ships.  Second,  as  discussed  by  Kanfer  and  Borman  (1986)  the  broad  delinea¬ 
tion  of  personal  history  variables  implies  that  multiple  factors  may  under¬ 
lie  biographical  predictors  (e.g.,  perhaps,  biographical  variables  are 
indexing  personality  and  interest  constructs).  Aggregating  across  bio¬ 
graphical  variables  tapping  different  underlying  factors  would  attenuate 
effects  of  these  predictors.  The  third  problem  relates  to  the  fact  that 
correlations  between  individual  biodata  predictor  items  and  performance  are 
often  reported.  Because  single  items  are  expected  to  be  less  potent  psy- 
chometrical ly  than  a  scale  of  items,  low  correlations  do  not  preclude  the 
predictive  validity  of  biodata  and  may  instead  indicate  a  need  for 
multiple-item  biodata  scales.  Further  research  is  therefore  needed  to 
explicate  relationships  between  biographical  and  other  variables  and  be¬ 
tween  biographical  variables  and  performance  before  biographical  items  can 
be  dismissed  as  good  predictors  of  sales  success. 

Personality  Variables 

Personality  measures  are  designed  to  index  traits--stable  non-cogni- 
tive  attributes--that  influence  behavior.  In  sales  and  recruiter  research 
the  validity  of  traits  such  as  empathy,  self-esteem,  and  dominance  for 
predicting  performance  has  been  investigated. 

Recruiter  research.  A  number  of  personality  variables  typically  pro¬ 
duce  significant  correlations  with  recruiter  performance  or  production 
criteria  (e.g.,  Atwater  et  al . ,  1986;  Borman  et  al . ,  1979,  1980,  1983; 

Krug,  1972;  Larriva,  1975).  Personality  inventories  most  often  used  in 
these  studies  are  the  Special  Assijnment  Battery  and  the  16PF.  Items 
tapping  characteristics  such  as  dominance,  aggressiveness,  self-confidence, 
and  spontaneity  appear  to  be  most  consistently  associated  with  recruiter 
performance.  Surgency,  cooperativeness,  and  orderliness  have  shown 
relationships  with  training  criteria,  but  not  with  supervisors’  ratings  of 
job  performance  (Massey  &  Mullins,  1966).  Likewise,  empathy,  achievement 
motivation,  and  sociability  were  not  related  to  performance  in  a  study  by 
Brown  et  al .  (1975) . 

Civilian  sales  research.  Aggregating  across  personality  constructs 
and  across  types  of  sales  jobs,  Churchill  et  al .  (1985)  obtained  a  mean 
corrected  correlation  of  .26  between  personality  and  motivation  variables 
and  sales  performance.  When  individual  personality  traits  are  examined  sep¬ 
arately,  certain  results  across  studies  appear  quite  consistent.  For 
example,  personality  scales  tapping  dominance  and  forcefulness  are  often 
associated  with  civilian  sales  performance  (e.g.,  Dunnette  &  Kirchner, 

1960;  Harwell,  1960;  Lamont  &  Lundstrom,  1977;  Rodgers,  1956).  Results  for 
self-esteem  variables  have  been  positive  (Bagozzi,  1980;  Ghiselli,  1969); 
however,  nonsignificant  findings  are  also  present  (Gable,  Mattheiss,  & 
Muczyk,  1984).  Empathy  measures  (e.g.,  Cotham,  1969;  Lamont  &  Lundstrom, 
1977)  and  achievement  motivation  variables  (e.g.,  Bagozzi,  1980;  Ghiselli, 
1969;  Matteson,  Ivancevich,  and  Smith,  1984)  have  both  provided  mixed  re- 


suits.  Items  designed  to  tap  supervisory  ability  were  significant  predic¬ 
tors  in  one  study  (Ghiselli,  1969)  and  initiative  produced  positive  results 
in  another  (Gable  et  al.,  1984).  Social  poise  as  indexed  by  the  Minnesota 
Multiphasic  Personality  Inventory  (MMPI)  K  scale  has  also  yielded  a  posi¬ 
tive  relationship  with  performance  (Ruch  &  Ruch,  1967). 

Summary.  Recruiter  and  civilian  salesperson  performance  both  appear 
to  be  related  to  personality  attributes.  More  specifically,  variables 
indexing  dominance  and  forcefulness  appear  to  be  related  to  both  recruiter 
and  salesperson  performance.  Self-esteem  and  confidence  have  yielded 
positive  relationships  in  both  recruiter  and  civilian  sales  contexts. 

Empathy  and  achievement  motivation  variables,  on  the  other  hand,  do  not 
appear  to  be  highly  associated  with  performance  in  either  arena. 

Aggregating  findings  across  personality  traits  and  across  types  of 
sales  jobs  can  have  complex  and  attenuating  effects  on  predictor-criterion 
relationships.  For  instance,  product  and  client  type  were  found  to  be 
strong  moderators  of  personality/motivation  predictor-criterion  relation¬ 
ships  in  the  Churchill  et  al .  (1985)  meta-analysis.  Indeed,  it  is  possible 
that  personality  variables  are,  in  general,  more  or  less  predictive  depend¬ 
ing  upon  the  type  of  sales  job  being  studied.  It  is  also  possible  that 
specific  personality  traits  do  underlie  performance  in  most  sales  jobs 
while  other  traits  are  related  to  performance  in  specific  jobs  or  are  not 
related  to  sales  performance  at  all.  Combining  personality  variables  does 
not  explicate  these  relationships. 

Vocational  Interest  Variables 

Recruiter  research.  Vocational  interest  variables  have  proven  to  be 
good  predictors  of  both  recruiter  performance  (cf.  Abrahams  et  al . ,  1973; 
Borman  et  al . ,  1979;  1980;  1983;  Graf  &  Bower,  1976;  Wollack  S  Kipnis,  1960) 
and  recruiter  production  (Atwater  et  al . ,  1986).  Many  of  these  studies 
have  involved  development  of  empirical  keys  of  valid  items,  and  some  have 
not  discussed  the  type  of  items  characterizing  the  key.  Items  referring  to 
interest  in  extroverted,  dominant,  or  competitive  activities  and  interest 
in  politics  and  law  are  examples  of  those  related  to  criteria  when  types  of 
valid  items  are  reported  (cf.  Atwater  et  al . ,  1986;  Borman  et  al . ,  1979, 
1980,  1983;  Wollack  &  Kipnis,  1960).  The  Strong-Campbell  Interest  Inven¬ 
tory  and  the  Kuder  Preference  Record  are  the  interest  inventories  most 
often  employed  in  recruiter  selection  research. 

Civilian  sales  research.  As  discussed  by  Kanfer  and  Borman  (1986) 
civilian  sales  research  has  focused  on  comparisons  of  response  patterns 
across  types  of  sales  jobs  rather  than  using  interest  variables  to  predict 
performance.  The  validation  research  that  does  exist  suggests  that  the 
type  of  sales  job  may  influence  the  predictive  validity  of  vocational 
interest  measures  (Dunnette  &  Kirchner,  1960). 

Summary.  While  there  is  a  relative  dearth  of  research  exploring 
predictive  validity  of  vocational  interest  measures  against  civilian  sales 
performance,  interest  variables  have  been  investigated  frequently  and  with 
success  in  military  recruiter  research. 
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Skill  Level  Variables 


Skill  level  measures  are  designed  to  assess  the  individual’s  current 
level  of  functioning  with  respect  to  specific  job  activities  and  are  thus 
often  developed  specifically  for  the  sample  under  investigation. 

Recruiter  research.  The  only  recruiter  selection  study  investigating 
the  utility  of  skill  level  variables  was  conducted  by  Brown  et  al.,  1975. 
Recruiters  were  asked  to  simulate  a  presentation  of  the  benefits  of  Army 
life  to  a  prospective  enlistee.  Presentations  were  recorded  and  scored. 

The  scores  differentiated  significantly  between  two  (high  and  low)  crite¬ 
rion  groups.  Skill  level  variables  were  evaluated  against  training  crite¬ 
ria  in  two  studies  by  Borman  and  his  colleagues  (Borman,  1982;  Borman  et 
al .  1983).  In  both  studies,  performance  on  four  assessment  center  exer- 
cises--cold  call  with  prospect,  follow-up  interview  with  prospect,  five 
minute  speech  about  the  Army,  and  in-basket--was  significantly  correlated 
with  training  criteria. 

Civilian  sales  research.  Interpersonal  skills  such  as  the  ability  to 
identify  verbal  and  non-verbal  cues  (Weitz,  1979),  to  maintain  eye  contact, 
and  to  produce  a  good  impression  (Pace,  1962)  have  shown  some  association 
with  sales  performance.  Churchill  et  al .  (1985)  reported  a  corrected  mean 
correlation  of  .32  between  skill  level  predictors  and  performance.  The 
authors  also  found  these  predictors  to  be  less  influenced  by  the  product 
and  client  type  than  are  personality  and  biographical  variables. 

Summary.  The  fact  that  little  recruiter  selection  research  has  in¬ 
vestigated  the  utility  of  skill  level  predictors  limits  comparison  of 
military  recruiter  and  civilian  sales  arenas.  The  Borman  et  al .  (1982, 
1983)  research  does  suggest  that  assessment  centers  are  valid  predictors  of 
training  criteria;  however,  the  utility  of  these  variables  for  predicting 
recruiter  performance  on-the-job  has  not  been  explored.  Research  in  the 
civilian  sector  does  suggest  that  interpersonal  skill  variables  may  have 
utility  for  predicting  job  performance. 

Aptitude  Variables 

Recruiter  research.  General  aptitude  and  verbal  ability  measures  have 
yielded  little  or  no  relationship  with  recruiter  performance  criteria.  For 
example,  neither  of  two  verbal  ability  tests  employed  by  Massey  and  Mullins 
(1966)  predicted  performance  ratings.  Similarly,  only  one  out  of  24  verbal 
ability-performance  relationships  was  significant  in  a  study  by  Wollack  and 
Kipnis  (1960).  Aptitude  measures  of  general  intelligence,  arithmetic  rea¬ 
soning,  and  mechanical  comprehension  have  also  shown  no  relationship  with 
performance  criteria  (cf.  Brown  et  al . ,  1975;  Wollack  &  Kipnis,  1960). 

Civilian  sales  research.  Aptitude  measures  have  met  with  little  to 
mixed  success  as  predictors  of  sales  criteria.  For  instance,  Harrell 
(1960)  and  Bray  and  Campbell  (1968)  both  found  general  ability  measures  to 
be  correlated  with  sales  performance.  Miner  (1962),  however  found  that  the 
only  Wechsler  Adult  Intelligence  Scale  correlating  with  an  objective  crite¬ 
rion  was  the  arithmetic  subtest.  Rush  (1953)  found  that  business  arithme¬ 
tic  and  college  grades  were  positively  correlated  with  an  objective  crite¬ 
rion.  Verbal  intelligence,  on  the  other  hand  has  correlated  negatively 
with  sales  success  (Bagozzi,  1980).  Of  the  various  person  attribute  pre- 
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dictors  investigated  in  the  Churchill  et  al .  (1985)  meta-analysis,  aptitude 
measures  produced  the  lowest  predictor-criterion  correlation  (attenuation 
corrected  r  =  .19),  and  no  moderating  effects  due  to  product  or  client 
types  were  observed. 

Summary.  In  general,  aptitude  measures  have  shown  little  or  no  util¬ 
ity  for  predicting  recruiter  or  civilian  sales  performance. 

Summary:  Predictor-Criterion  Relationships 

Our  review  of  military  recruiter  and  civilian  salesperson  selection 
research  suggests  that  there  are  two  major  similarities  between  observed 
predictor-criterion  relationships.  First,  both  recruiter  and  salesperson 
performance  appear  to  be  related  to  personality  attributes.  In  particular, 
personality  variables  indexing  dominance  and  forcefulness  are  likely  to  be 
associated  with  recruiter  and  salesperson  performance.  Other  personality 
variables  appear  to  yield  less  consistent  relationships.  It  is  possible 
that  the  validity  of  specific  personality  variables  is  moderated  by  the 
type  of  sales  job.  To  date,  research  does  not  clarify  this  issue.  The 
second  major  similarity  is  the  lack  of  support  for  aptitude  measures  as 
predictors  of  either  sales  or  recruiting  performance. 
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Summary  of  the  Development  of  the  Special  Assignment  Battery  (SAB) 

Over  the  years,  extensive  research  has  been  conducted  to  delineate  the 
personal  attributes  related  to  recruiter  success.  A  number  of  these 
studies  have  employed  a  specific  battery  of  vocational  interest,  biogra¬ 
phical,  and  personality  items  called  the  Special  Assignment  Battery  (SAB). 
Because  the  SAB  has  proven  successful  as  a  predictor  of  recruiter  perfor¬ 
mance  in  several  research  efforts  and  appears  to  be  a  good  starting  point 
for  future  recruiter  selection  research,  this  appendix  summarizes  its 
development  and  subsequent  use  in  research. 

The  development  of  the  SAB  began  with  a  job  analysis  wherein 
researchers  learned  about  the  recruiter  job  and  developed  hypotheses  about 
the  personal  characteristics  (e.g.,  personality,  interests,  background) 
related  to  recruiter  performance  (Borman,  Hough  &  Dunnette,  1976).  Through 
numerous  interviews  and  workshops  with  recruiters,  the  research  team  also 
developed  behavior-based  performance  appraisal  rating  scales  for 
recruiters.  Ratings  of  recruiter  performance  on  these  rating  scales  have 
served  as  the  criteria  for  evaluating  the  validity  of  the  SAB  in  a  number 
of  studies  and  were  instrumental  in  the  development  of  the  SAB.  We, 
therefore,  discuss  the  development  of  the  rating  scales  briefly  in  the 
following  paragraphs  and  then  summarize  the  development  of  the  SAB  itself. 

Development  of  Behavior-Based  Recruiter  Rating  Scales 

To  develop  behavior-based  rating  scales  for  recruiters  we  used  the 
behavior  analysis,  or  critical  incident  methodology.  Over  800  examples  of 
effective  and  ineffective  recruiter  performance  were  gathered  from  field 
recruiters,  their  supervisors,  and  recruits.  Performance  categories  were 
then  developed  on  the  basis  of  the  content  of  the  incidents,  and  recruiters 
and  their  supervisors  categorized  the  incidents  and  rated  the  effectiveness 
each  represented.  Finally,  Behavior  Summary  Scales  (BSS)  were  developed  by 
grouping  reliably  sorted  and  rated  incidents  into  each  dimension  and 
writing  behavior  summary  statements  reflecting  the  content  of  those  inci¬ 
dents. 

The  resulting  Navy  recruiter  scales  consisted  of  eight  dimensions  or 
categories  of  performance.  They  are: 

1.  Locating  and  Contacting  Prospects, 

2.  Gaining  and  Maintaining  Rapport, 

3.  Obtaining  Information  from  Prospects  and  Making  Good  Person- 
Navy  Fits, 


*In  1985,  the  Army  administered  the  Recruiter  Selection  Battery-Experimen¬ 
tal  (RSB-X)  to  over  400  recruiters.  This  battery,  the  RSB-X,  is  a  slight 
modification  of  the  SAB.  A  few  of  the  biographical  items  were  tailored  to 
the  Army,  and  the  interest  inventory  was  omitted. 
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4.  Salesmanship  Skills, 

5.  Establishing  and  Maintaining  Good  Relationships  in  the  Community, 

6.  Providing  Knowledgeable  and  Accurate  Information  about  the  Navy, 

7.  Administrative  Skills,  and 

8.  Supporting  Other  Recruiters  and  the  Command. 

In  order  to  evaluate  these  rating  scales,  supervisory,  peer,  and  self¬ 
performance  ratings  were  collected  and  analyzed  for  267  Navy  and  118  Marine 
Corps  recruiters  (Borman  et  al . ,  1979).  Interrater  reliabilities  obtained 
for  the  pooled  peer-supervisor-self-ratings  were  r  -  .57  for  the  Navy  and 
r  *  .48  for  the  Marine  Corps  scales.  Analysis  of  rating  errors  (i.e., 
leniency,  restriction-of-range,  and  halo)  showed  results  comparable  to 
those  discussed  previously  concerning  the  summary  scale  format.  When  the 
pooled  ratings  were  factor  analyzed,  a  three-factor  solution  resulted  as 
described  below: 

1.  Selling  Skills  -  selling  Navy  effectively  to  prospects;  displaying 
confidence  and  effectiveness  in  the  recruiting  sequence  - 
prospecting,  selling,  and  closing; 

2.  Human  Relations  Skills  -  establishing  and  maintaining  good 
interpersonal  relations  with  prospects,  recruits,  and  persons  in 
the  community;  and 

3.  Organizing  Skills  -  planning  ahead  and  organizing  time 
efficiently,  completing  paperwork  accurately  and  on  time. 

Subsequent  research  with  these  scales  (Borman  et  al.,  1981)  replicated 
the  three-factor  solution  shown  above.  Therefore,  the  scales  appear  to  be 
reliable,  robust  to  various  rating  errors,  and  representative  of  a  stable 
and  psychologically  meaningful  factor  structure. 

Development  of  the  Special  Assignment  Battery 

Hypothesized  (and  later,  empirical)  relationships  between  the  perfor¬ 
mance  factors  described  above  and  personality,  interest  and  background 
items  were  used  to  formulate  the  SAB  and  its  keys.  As  shown  in  Figure  A-l 
hypothesized  links  were  first  used  to  select  items  and  scales  from 
published  inventories,  and  item-performance  correlations  from  pretest  data 
were  used  to  further  refine  the  item  pool.  At  this  point,  four  keys  were 
developed  for  the  SAB  (i.e.,  one  for  each  performance  category  that  the 
items  were  expected  to  predict).  A  concurrent  validation  study  was  then 


2The  SAB  keys  are  named  after  the  performance  category  they  are  intended  to 
predict.  Validity  estimates  are  correlations  between  the  performance  con¬ 
struct  (e.g.,  Selling  Skills)  and  the  SAB  key  (e.g.,  Selling  Skills) 
designed  to  predict  that  aspect  of  performance. 


conducted,  and  the  keys  were  cross-validated  via  a  Monte  Carlo  procedure 
(Borman,  Toquam,  and  Rosse,  1979).  Next,  the  items  within  each  key  were 
factor  analyzed  to  identify  the  individual  differences  constructs  tapped  by 
each  key,  and  new  items,  targeted  toward  tapping  the  same  non-cogniti ve 
constructs  were  written.  In  turn,  the  new  items  along  with  the  old  items 
were  validated  in  a  concurrent  study  (Borman,  Rosse,  and  Abrahams,  1980), 
and  final  keys  were  developed. 

Research  Employing  the  SAB 

Subsequent  research  employing  the  SAB  can  be  categorized  into  two 
types:  (1)  developmental/refinement  studies  tailoring  or  examining  SAB 
items,  and  (2)  attempts  to  validate  SAB  scores  against  various  criteria. 

Developmental  research.  In  one  study  (Borman,  Rosse,  &  Rose,  1983) 
the  SAB  items  were  refined  and  tailored  to  the  Navy  officer  recruiter  job. 
Likewise,  one  goal  of  the  Army’s  ongoing  research  is  to  examine  Army 
recruiters’  SAB  scores  (RSB-X)  and  compare  them  to  those  of  Navy 
recruiters. 

Validation  research.  Recruiter  performance,  production,  and  training 
success  have  served  as  criteria  for  validating  the  SAB.  Table  A-l  presents 
the  validity  coefficients  obtained  in  these  studies.  In  sum,  the  SAB  has 
consistently  predicted  recruiter  performance  (i.e.,  self-supervisor-peer 
ratings)  across  studies. 

Attempts  to  validate  the  SAB  against  a  production  criterion  have  also 
been  successful.  In  one  study  the  SAB  validated  poorly  against  production 
(Borman  et  al . ,  1979).  This  finding  is,  however,  inconsistent  with  later 
research  (Atwater,  Abrahams,  &  Trent,  1986;  Borman,  Rosse,  &  Abrahams, 
1980). 

In  two  studies  the  SAB  has  not  validly  predicted  recruiter  success  in 
training.  Thus,  it  appears  that  the  individual  differences  constructs  that 
lead  to,  or  predict,  recruiter  performance  and  production  on-the-job  differ 
somewhat  from  those  characterizing  recruiters  who  are  successful  in 
training.  Indeed,  the  behaviors  that  make  for  success  in  training  (e.g., 
studying)  may  be  quite  different  from  those  leading  to  job  success.  It  is 
likely  that  cognitive  variables,  as  well  as  non-cogniti ve,  are  important 
for  training  performance,  and  the  validation  of  assessment  center  exercises 
tapping  both  cognitive  and  non-cognitive  constructs  against  training 
criteria  (Borman,  1982;  Borman,  Eaton  &  Rosse,  1983)  supports  this  view. 


Table  A-l,  Summary  of  SAB  Validity  Estimates  Across  Studies 

Ratings  Criteria 


SAB  Scales/Performance  Category 


1. 

Navy  267 

Personal ity 

21 

— 

16 

29 

21 

MC  118 

19 

19 

38 

24 

Navy  267 

Interest 

19 

17 

26 

17 

-  - 

MC  118 

19 

24 

20 

26 

Navy  267 

Biographical 

16 

05 

23 

23 

MC  118 

-01 

10 

15 

15 

-  _ 

2. 

Navy  194 

Old  plus  New 

Personal i ty 

Old  plus  New 

23 

24 

15 

22 

-- 

-- 

Interest 

21 

22 

12 

22 

.  . 

Final  Keys 

Personal i ty  & 

Interest 

43 

46 

40 

43 

3. 

Navy  132 
Officer 

Personal i ty 

15 

30 

18 

19 

Recruiter 

Interest 

16 

11 

07 

16 

5. 

MC  1006 

Final  Keys 

Personal i ty  & 

Interest 

19 

15 

13 

20 

20 

Production  Crit 

eria _ 

1. 

Navy  267 

Personal ity 

01 

MC  118 

-- 

-- 

24 

Navy  267 

Interest 

-02 

MC  118 

-- 

10 

Navy  267 

Biographical 

01 

MC  118 

-- 

08 

2. 

Navy  194 

Final  Keys 

22 

23 

13 

26 

27 

5. 

MC  1005 

Final  Keys 

23 

15 

09 

24 

22 

MC  315 

Final  Keys 

23 

15 

09 

24 

22 

.  -- 

Training  Criteria/F 

nal  Ke\ 

s 

Criterion 

4. 

Army  50 

Phase  1  Performance 

02 

-06 

-01 

-09 

Army  46 

Phase  2  Performance 

17 

15 

07 

15 

Army  46 

Time  to  Complete 

-12 

-27 

-15 

-09 

-- 

5. 

MC  431 

Completion/Failure 

*  “ 

“  * 

“  " 

08 

Studies 


1.  Cross-validities  obtained  via  Monte  Carlo  procedure  by  Borman,  Toquam,  and  Rosse 
(1979). 

2.  Borman,  Rosse,  and  Abrahams  (1900);  final  key  validities  were  not  cross- 
val  idated. 

3.  Borman,  Rosse,  and  Rose  (1983). 

4.  Borman  (1982). 

5.  Atwater,  Abrahams,  and  Trent  (1986). 
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